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1974 PROPOSAL TO ARPA 

SUTtlARY 

The results of a decade of uork on Artificial Intelligence have brought 
ue to the threshold of a neM phase of knowledge-baaed programming — In 
which ue can design computer system* that 

(1) react reasonably to significantly complicated sltuatlone and 

<2) perhaps more important for the future — Interact 
Intelligently ulth their operators when they encounter 
1 imi tat lone, bugs, or Insufficient information. 

This proposal lays out programmes for bringing several such eystems near 
to the point of useful application. These Include: 

A physical "micro-automation - system for maintenance and repair 
of electronic circuits* 

A related "expert* problem-solving program for diagnosis and 
modification of electronic circuits. 

A aet of advanced "Automatic Programming" techniques and eystene 
for aid in developing and debugging large computer programs. 

Some Advanced Natural Language application methode and eystems 
for use uith these and other interactive projects. 

A eeriee of specific "expert - problem solvers. Including Cheee 
analysis. 

Steps touard a new generation of mors intelligent Information 
Retrieval and Management Assistance eystems. 

The application areas ere chosen to advance our general competence, 
clarify dark areae, and provide Morking prototypes that should be 
especially helpful in bringing other areas to the same practical stages. 
The proposal details plans for two years research uork, beginning 
January 1974, and ue further draft uhat ue believe must be done to bring 
each area to the point of major practical applications In the order of 
five years. 



OVERVIEW OF THIS DOCUMENT 

This proposal fallows mora than usually quickly cur previous proposal of 
barely six months ago. Thus, much of this proposal is for continuation 
of work already undsr way. However, us can now specify more precisely 
the problems and milestones us expect to encounter and achieve, and more 
precise assignments of people and resources in the laboratory to 
specific subtaeks. This Is in response to the considerable increase In 
size and complexity of the projects, as compared to those of the past 
which were usually attacked by individuals rather than by groups. 

SECTION 1 presents our work-plan for the next two years fplusl, 
classifying the tasks into seven areas of concentration. Each area ie 
further broken down into one or two primary topic and a number of 
secondary topics. Each primary topics has clear-cut milestone goals for 
the two year period. Those goals mentioned in Section 1 are selected to 
illustrate the nature of the problem area; they are real goals but not 
necessarily the most important or most difficult. 

These areas have discrete milestones and idsntifiable personnel with 
reeponeibi I i ty to those goals. However* to some extent these 
assignments are nominal. In that there is a great deaf of interaction 
and sharing of expertise within the laboratory. In other words. Section 
1 presents a "linear" model of the laboratory; listing seven "terms" as 
though independent and ignoring the Important interactions. 

PROJECTS 1 through 7 are the body of the proposal. Each states the 
goals of one of our application areas, its problems, proposed methods, 
resources needed, expected "milestones", and the people who will be 
responsible. 

PROJECT 1 describes in detail the propossd Visual Electronic Repairman 
Project, which includes the goals of our previous work on flicro- 
Automation and Machine Vleion. 

PROJECT 2 describes the "Knowledge-Based Electronic Repairman" 
project. 

PROJECT 3 describes our Automatic Programming, Dsbugglng and Self- 
Documentation research programme. 

PROJECT 4 describes our work toward Natural Language interactive 
semantic systems. 

PROJECT 5 describee a number of theoretical projects concsrning 
representation of knowledge, learning, logic, etc. 



PROJECT G describes the Chess project and several other small "export 
problen solving" projects* 

PROJECT 7 proposes two study projects concerned with Improving the etatl 
of the art In Information Retrieval and In Management Assietanct. 

SECTION 3 discusses the state of A. I., In terns of methods, outstanding 
problems, and probable tims-scafes. It also explains the coheeiveneee 
of the entire area of study. 

SECTION 5 it the A.I. Laboratory Bibliography. 



SECTION 1 
SCHEMATIC OUTLINES OF THE PROJECTS 



1. Topic* with emphasis on VISION AND MANIPULATION 
PRIMARY TOPIC: Circuitboard Repair Robot 

SECONDARY TOPICS: Manipulating liquids 

Seeing Irregularly shaped forme 
Mand-eue coordination 
Micro-Automation development 
Inexpensive equipment for research 

2* Topics Mith emphasis on DEBUGGING ELECTRONIC CIRCUITS 

PRIMARY TOPIC: Electronic Trouble Shooting 

SECONDARY TOPICSt Syntax and Semantics for circuits 

Diagnosis of faults 
Planning of signal tracing 
Planning of repair 

3* Topics Mith enphasis on AUTOMATIC PROGRAMMING ANO DEBUGGING 

PRIMARY TOPICS: Debugging Electronics and Graphics Programs 

Imp lamentation of ACTOR formalism 

SECONDARY TOPICSi Classification of common program bugs 

Intention-Orisnted Automatic Programming 
Automatic Annotation and Self-Documentation 



4. Topics with an emphasis on NATURAL LANGUAGE 

PRIMARY TOPICS: Semantic theories of Syntax (with U. Martin) 

V. Pratt's syntax language development 

SECONDARY TOPICS: Representation of extended events and scenarios 

Interfaces for other projects 



S. Topic* with a general THEORETICAL EMPHASIS 



PRIMARY TOPICS! 



Reprseentation of knowledge* Frames, Actore 
Claeslf icatlon of Common-Sense Knowledge 



SECONDARY TOPICS; Inductive Inference ISolomonoff) 

Mathematical Complexity 
Modal Loglca (Getter) 
Phye I o l og \ ca I Theor I ee (Marr ) 



6. Topics with an emphasis on EXPERT PROBLEM SOLVING 



PRIMARY TOPICS: 



Chess and Surprise Analysis (Greentolatt) 
Geometry (Broun) 



SECONDARY TOPICS: Theorem-Proving (Nevlns) 

Qual i tative Physics 
Decision under uncertainty 
Advanced Learning machines 



7. Topics concerned with IDENTIFYING AREAS OF APPLICATION 



SUB-TOPICS: 



Advanced Automation; Micro-Automation 

Advanced Information Systems 

Heuristic Information Retrieval 

Personal Management Assistants 

Undersea and Space 

Overview and Assessment of Problems in AI 



SECTION 2 
nc PROJECTS 

Thie section is the body of the technical proposal. It is subdivided 
Into seven PROJECTS. Each subsection begins with a compact overview 
with, more or less, the following formati 

DEFINITION: A brief explanation of what the project is about. 

niLESTONES: Ue divide each project into phases as appropriate. 
Milestones are given for each phase, Ue give projections beyond the 
two-year proposal period to show hou ue envision these projects coning 
to fruition over the subsequent fsw years- 
It is understood that this Is a very difficult field, our goale are 
ambitious, and our standards are extremely high. Ue have tried to be 
realistic about these estimates but sone nay take longer. The overall 
hope is to get each of these areas onto eolid foundations — in the 
research and prototype sense — within three years, so that 
dissemination of techniques and equipment can put them well on the road 
to practical exploitation within five years* 

APPLICATIONS: Each subssction summarizes briefly some applications of 
the project to Important problems. 

PROBLETISt Ue mention the outstanding difficulties and bottlenecks that 
appear to be the most serious* 

COSTSt The especially expensive aspects of each project. Only m few 
require "special" equipment, but all except the most 
theoretical areas require unusually heavy computational 
services under large-memory, time-shared operating systems. 

PERSONNEL: The principal innovators and people responsible for results* 
and their associates as Known at this tine* 



Acknowledgement! Much of the text of the following sectione is adapted 
from drafts by P. Uinston and B. K.P.Horn Project It G.J.Sussman, Project 
2i I. Goldstein, Project 3*1* C. Hewitt, Project 3.2, V. Pratt, Project 
4; R. Greenblatt, Project 6. 



PROJECT 1 
VISION, MANIPULATION. niCRO-AUTOtlATION 
The Physical Electronic Repairman 

DEFINITION: Ur uant to bring machine vision to the point where one 
could feel comfortable about saying that the computer "can see". Ae 
emphasized In our earlier work, this is not a we I l-def Ined, isolated 
task, because perception in general, and vision In particular, cannot be 
cut away from general knowledge and intelligence. But within the 
microworld of the Electronic Repairman, we can define what it means to 
eee, and many ueeful applications are within reach, 

Ue already can get the computer to scan and "understand" we I I -control led 
ecenes involving objects with neat geometric shapes, some aepecte of 
etandard printed circuits, and some moderately complicated shadow 
el tuat lone* 

Specifically, thie project will be concerned with real-world vleual 
analyele of the klnde of scenes found inside electronic assemblies. 

niLESTONES: 



JULY 1974 
OEC* 1374 

DEC. 1975 



Visual analysis 
Recognition of i 



of printed circuit layouts* 
ilectronlc components 



Mechanical inspection of solder jolnte. 
Use of test probes at circuit points 
Use of Miniature Hand-Eye system 
Pouring liquids with visual control 

Vieuat inspection of solder joints. 

Identification or verification of physically broken part 

Diagnosis of simple malfunction 

Physical replacement of defective components. 

Connection to electronics debugging programs [Project 21 



OEC- 1977 Connection with NATURAL LANGUAGE ASSISTANT, 

Connection with Automatic Programming Assistant 
Correlation of physical componsnt with verbal description. 
Assembly of a Mhole kit. 

The latter ie also an ambition of the Stanford Project, and cloee 
collaboration should be possible by that time. 



APPLICATIONS: Assembly, Inspection, maintenance, and repair of 
computers and other electronic equipment* 

Generalization to similar functions of small mechanical assemblies. 

Servicing of electronic assemblies of larger systems, e.g.* electrical 
systems of engines, antennae, undersea and space devices. (Eventually) 
Maintenance of sub-miniature systems. 

Computer vision has extenetve applications to supplementing other 
eysteme. For example, such a system could maintain surveillance over 
the area around a dangerous machine, a secure installation, an 
Intensive-care patient, an object-in-road-ahead Marning. 

ENGINEERING PROBLEMS: 

Curved objects Representation of component** 

Shading and Texture, Direct range-finding. 

Arm and hand design. Arm dynamics. 

Force- feedback sensation. Motion tracking- 
Motion parallax. 

Qualitative physics of components, wires, etc. 
Understanding shading and texture 

Developing descriptive languages for representing components. 
Developing a language for convenience to human operators. 
Heterarchical programming for real-time interrupt environment. 

COSTS: Use of very large programming systems and memory 

Diagnostic display 

Completion of Micro-Automation Laboratory. 

PERSONNEL: Prof* Minston (vision system development and software) 

Prof. Horn (hardware and software development) 
Research Staff, Consultants, Students (see below) 



TECHNICAL ASPECTS OF PROJECT 1 
RESEARCH IN MACHINE VISION 
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INTRCOUCTICH 

Uork on machino vision has progressed rapidly in the 
nany basic Issues are non more eharply defined, perm 
outeide the restricted uorld of carefully prepared e 



laat three yeare. 
tting ue to focus 
nple polyhedra* 



At the "performance 1 level, ue can take a col lection of flat-sided 
objects of assorted shapes, pi Is then in a disorderly heap, and aek 
the program to analyse, disassemble, and rearrange the objects into 
another, orderly structure. The latter can be specified by a 
symbol lc description or by presenting a physical example to be 
analysed by the system. Many H lou-level p vieion problems had to be 
eolved to reach this level of performance, flany of them are 
summarized in our January, 1972 Progress Rsport, and much more 
detail Is available In technical notes and reports. 

It Is Important to note that ue have made no compromises In our 
original long-term goal to est a firm foundation for Monocular 
machine vision! This vision system uorks as well on Pictures of a 
scene as it does on the physcial scene itself* It is not based 
essentially on the use of physical range-finding eethods 
probe exploration, or other "active" sensore. 



tacti le- 



This is not to say that active sensors are not valuable! Ue plan 
many uses of them in our project. Ue sinply want to underline the 
scientific importance of the systematic work on what one might call 
picture-vision, because casual onlookers eight be unduly impressed 
111 th now easily one can get superficially similar practical 
applications by more application-tailored methods- But the 
understanding that has cone froa the study of the pure, monocular 
vreion problem is a more solid and permanent addition to our 
"general" capabilities, both practical and scientific. This 
knowledge will always be available when active eysteme run Into 
difficulty because of (for example) large distances, power 
limitations, monitoring through TV type sensors, need for not 
disturbing the scene, etc* And perhaps most important, thie uork 
has already made outstanding contributions to »odern cognitive 
psychology in In understanding human vision. See. for example, the 



widely used Introductory psychology textbook of Donald Norman* or 
the assessment of Sutherland in the British SRC report on 
Artificial Intelligence research (sea Section 3). 

Problems remain, to be sure, but now that the construction of a multi- 
purpose Blocks-Uorld hand-eye system is behind us, it is time to 
reorient our efforts towards richer domains. 

First, ue need to expand our basic features to Include texture, 
color, shading, sharpness of focus, hlghltghte, shadows, and 
■ot ion- 
Second, ue uant to study visual situations In which perceived 
context can have a substantive role in analysis* 

Third, ue plan to extend the Interaction with other concentration 
areas around the laboratory so as to profit fro* advances In 
natural language, representation of knouledge, problem solving, 
advanced programming developments* 

B. PROGRESS TO DATE 

Before outlining our position with respect to further work, ue uleh to 
describe a few recently completed projects that sees likely to support 
new studies, 

1. David Ualtz has worked out a semantic theory of polyhedral Una 
drawings that ia a major breakthrough In several respects. The 
theory gives deep Insights into the success of earlier uork and 
provides a powerful analysis capability for separating regions into 
bodies; in identifying edges as convex, concave, obscuring, shadow 
or crack: in using shadous to determine contact; and in reasoning 
out tha orientation of object faces. That aore information should 
simplify problem solving Is obvious; Ualtz has gone far beyond tha 
truism and ehoun how the idea can be worked out using a formal iem 
and representation structure that should contribute to uork in 
advanced systems for both visual and linguistic work. In all early 
vision projects, shadow boundaries caused malfunctions because they 
were often interpreted as physical boundaries! in Waltz* eyetam 
they are exploited In sevaral ways to correct other kinds of errors 
and ambiguities, evsn to asserting that a missing Una must exiet 
and should be looked for more carefully. 

2* Previous vision systems suffered from an artificial division 
into I ine-f indor/scene-ana lysis partnerships, communicating only by 
way of a handed-over line drawing. The new systems of Jerry Lerman 
and Yoshlakl Shlral show how the barrier can be eliminated and how 
high level knowledge of physical constraints and partial analysla 
can guide the flltere and trackare that most intimately deal with 



low-level intensity information. The systems ere thus prime 
examples of the heterarchical programing concept discussed 
elsewhere In this proposal- 
Brief |U| the problem is this. In older systems (as well as in 

older psychological theories of vision) the process was divided 

into steps, for example! 

1, Find distinctive visual feature points, e.g.* large 
gradients. 

2. Aggregate them into higher-level elements, e.g., lines. 
3* Aggregate those into. say t regions. 

4. Aggregate these into. say, "objects". 

5. Identify or "recognize" the objects. 

6. Aggregate the objects into familiar structures. 

Such systems worked. If at all, only on carefully prepared scenes 
and "toy" problem demonstrations. Ths trouble Is that there are eo 
many places for errors, and these propagated so mercilessly* that 

the chance of the whole chain working was too small to be useful* 
In the new systems we have found ways to use knowledge at a high 

level — say, about what kinds of edges occur in shadowed* concave 
regions, to continually monitor the performance of the low level 
" I ine^f inders" operating at the primitive "scanning" level. 

3. Tin Finin has given the evolving vision system considerable 
deductive depth through several goal-oriented programs. One of 
these specializes in using a thsory of "perceived groups' 1 . Often. 
some of an object's Individual dimensions, position, or orientation 
parameters are indeterminate because of an obstruction in the line 
of sight. In these situations the vision system hypothesizee the 
missing information, using other objects considered elmilar by 
virtue of alignment in a stack, a common purpose, or simple 
proximity. This is one entry into the area of context driven 
analysis. 

4. Finin, Lerman, and Slesingsr havs completed a visual feedback 
module that checks the position of a block after positioning by the 
hand. Then it jiggles It into place If Its positional error 
exceeds a small threshold. This feedback link makes possible 
exploiting the random-access capability of a programmable image 
acquisition system by looking only at points lying on a email 
circle around expscted vertex locations. Flnln and Lerman have 
also completed a touch feedback module for use In certain cases 
when a monocular image is inherently ambiguous. 



5* Bob Uoodham has done Initial complementary work on vieual 
motion tracking. As the first step In a coffee-pouring 
demonstration, he has worked out and compared several mechanisms 
for monitoring the rising level of coffee in a stylized cup. 



Although this is still In a demonstration phase and not integrated 
Into the system, ue believe the nechanism will extend smoothly to 
such skills as shadow-aided placement of delicate objects* 

6* Scott Fahlman has devised a construction planning system which 
solves problems in two distinct directions. First, three 
dimensional modelling skill has been developed in the form of 
sophisticated touch and stability tests. Second, in cooperation 
with the specialists in CONNIVER language, he has demonstrated the 
need for and use of advanced control and data base mechanieme. The 
system can plan fairly complicated constructions requiring 
temporary scaffolding supports. 

7. Rich Boberg has explored the problem of reversing the analysis 
process, that is, reconstructing a scene from an abstract 
description* Ue believe this Is the first step toward an 
automatic design system where the machine contains and uaes 
considerable common sense knowledge about the constraints inherent 
in a physical world. In the next section we will discuss how 

Ounlavay's work pushes still further in this direction. 

8. John Holterbach probed the problem of describing complex shapes 
through work on compllcatd, higher order polyhedra. His heuristic 
theory of projection shows how many objects can be sensibly 
decomposed Into basic shapss, modified by protrusions and 
Indentations. 

9. In another domain. Hark Adler has shown how to make progress 
toward solving the problem of line drawings with curves. In a 
style reminiscent of initial work on polyhedra, he has outlined an 
approach to the analysis of some highly constrained kinds of 
drawings* This should contribute conceptually to work on more 
general real vision, to diagram reading and manipulating services, 
and eventually to personal assistant systems in which sketches must 
supplement natural language commands that are more clearly 
explained graphically. 

C, PROPOSED RESEARCH IN MACHINE VISION 

The traditional approach to "low- level" vision has been to bring In 
familiar mathematics from linear systems theory and elsewhere, and use 
it in generalized form. Certainly texture has been a primary source of 
problems for people interested In Fourier transforms and statistics, and 
in our own laboratory Prof, Horn has applied the mathematics of partial 
differential equations to the problem of deducing three-dimensional 
shape from two-dimensional shading information. But while 
mathematically derived computations on picture data are important to 
understand, th© research does not always couple well with what one hopes 
to get out of an intelligent machine. Transformations of textured 



pictures do not sten to help much in identifying a eolid as wood, 
plaster, or carpet, Horn's mathode, while accurate in producing space 
curves on uniformly painted surfaces, still do not go far toward the 
hierarchical data structure which on the highest level says that a 
surface appears to be part of a sphere, cone, or cylinder. 

Uhat ue are after arm common sense qualitative theories of low level 
vision processing, that can exploit constraints easily expressed only at 
higher levels of description. 

Shirai's paper, n A Heterarchical Program for Finding Objects," It 
prototypical of uhat we would like to have come out of our new studies. 
In that paper he describes a program we believe to be the best available 
translator of picture arrays into line drawings. It is instructive to 
see uhy It Is so good. 

The program consists of a feature point detector, a line tracker, and a 
line proposer. Shirai's feature point detector employs only a simple 
differencing operation used widely before. But Shirai couples that 
differencing operation with a heuristic filtering program which checke 
the ehape of the contrast curve against a four-point quality check list* 
Although simple, these tests are not the sort of things one builds Into 
a uni form, post t ion independent, I inear f i I ter. 

The point is further illustrated in Shirai's exploitation of tracking 
and proposing programs about which classical know I edge- free Mathematical 
methods have nothing at all to say. Again we find lists of common sense 
facts about the demonstration universe which translate into goal- 
oriented programs that get the job done. 

It ie this qualitative, common sense spirit that we want to extend to 
I arger syetems. 

In this direction, Horn, Levin, and harr have undertaken a new etudy of 
color, asking how a machine might exhibit the sane insensi t tvi ty that 
man has when naming colors under varying Illumination spectra and 
incident light dietribut ion. The "retinex 1 * theory of Land is regarded 
as a strong first step. In essence, that theory argues for color naming 
on the basis of three independently derived lightness order ings. one 
each in the red, green, and blue. The lightness orderings in turn arm 
determined at region boundaries with no part played by slow drift In 
absolute intensity- Land arguas that the independence of the lightness 
dmterminat ions prevents shift In Illumination from distorting perceived 
color and further that the discounting of slow drift across facee 
prevents oblique lighting from disturbing the relative lightness 
measures. 

But at a detailed level ue believe more work is to be done. Land's 
lightness measurement algorithm involves an unsatisfactory random walk 
procedure that wanders about the scene seemingly In search of the 



brightest region against Mhich to normalize other regions. 

Ue hope to substitute a more reasonable algorithm that reflects the tuo 
dimensional nature of the problem. Horn is working on the details of m 
theory that involves a successive differencing and threshof ding, and a 
calculation resembling simple solution of resistive networks. The 
method may lend itself to implementation in simple parallel hardware. 

Ue believe this work wi 1 1 be important to robots, especially where color 
is often of first Importance In identification and where identification 

must be accurate under all eorts of varied natural and artificial 
i I lumination. 

Continuing this deliberate program of formulating in machine-usable form 
common-sense observations about vision. Winston is trying to develop a 
qualitative theory of how one can determine solid shapes from shading 
and highlight information. In earlier work, Horn has shown how surface 
line equations can be arrived at analytically under assumptions of 
uniform reflectance. Uhat Uinston Is after Is a qualitative theory of 
shading facts that allows direct hypothesis about the approximate shape 
suggested by a surface without recourse to that surface's borderline 
shape. And eventually, one would want to be able to eliminate Horn's 
uniform surface assumption in favor of using additional knowledge of 
plausible intrinsic colorations of surfaces* 

The theory, like Shirai's hsterarchical line finder, ie expected to take 
the form of a collection of procedural ly embedded facts relating the 
relative location of highlights, shading gradients, and light sources to 
the conclusions about ths three-dimensional nature of the sample. 
Informally, such ideas are well knowm the work of J. Gibson In 
particular is widely acclaimed as important in explaining human 
perception of three dimensional configurations outside (and often* even, 
Insldel of the range of focus or stereoscopic determination. But 
although the Ingredients of such theories have been available for a very 
long time, this will be the first attempt, ue believe, to weld them 
together Into a coherent (and useful) system. 

John Hollerbach is working on the complementary problem of devising a 
description system rich enough to represent real curved surface objects. 
He expects to build on his success with polyhedra In which description 

segments divide complex objects nicsly into protrusions. Indentations, 
and basic projections of simple plane figures. He is nearly finished 
with the first generalization, that of describing the myriad Juge, 
bowls, crocks, and amphorae that make up the world of archeologlcal 
pottery. Uhen finished, his system will describe these objects in 

terms like "high shoulders, "flarsd base," and *narrou neck," just as 
does a human special iet in ths field. 

Ue have two problems in learning how to introduce textured objects to 
the machine. The first is to separate texture boundaries from object 



boundaries between objects with the same texture. The eecond problem is 
how to uee texture to determine the orientation of a surface. This 1e 
an old idea, as seen in many papers by J. Gibson, but earlier work has 
not included detailed theories demonstrated by or potentially 
demonstrable by a machine- Once again, we expect the description 
problem to be a key focus in our approach. Uithout taking image apace 
into Fourier space ue mil Mant to study several potential processes for 
1) determining texture granule boundaries, 2) calculating texture 
granule features such as area, shape, and boundary character let ice, and 
3) noting differences between two textures as a prelude to relative 
orientation conclusions- This works with a project in which Eugene 
Freuder wants to bring the most sophisticated knowledge to bear on 
identifying real objects. Hie focus is on the interface between image 
information and world knowledge. He cites as inspirational the work of 
Hewitt on PLANNER, Sussman and flcDermott on CONNIVER, Uinograd on the 
semantic interface, and Ualtz on constraint exploitation. Ue believe 
thie work will uncover general ideas about problem solving in a 
heterarchical system and force a step forward in real world vision 
capabi I i ties. Michael Dunlavey has the equal ly di ff leu 1 1 job of 
understanding what he calle "interface knowledge"! that knowledge which 
ie required to understand how general concepts interact by delving Into 
the details of their descriptions until the level is reached where 
interaction takes place- Specifically, Dunlavey has chosen the 
demonstration world of construction with toy bricks. The general 
concepts ar^ notions like "wall," "door," "chimney," and 'roof." The 
interface knowledge concerns the description of each of theee in terns o 
their constituent repeated brick patterns- Gsnsralizatlon to deal with 
an important part of ths design of real houses, buildings, snipe, and 
other constructions seems smooth and continuous^ 



D. VISION niLESTONES 
The chart below summarizes our major vision actlvltiee. 

color Correctly nana the colors of randomly 

arrayed colored papers undsr a variety 
of illuminants. (Ue I I under way) 

Use color In conjunction with 

established i mags-process I ng 

techniques to Identify boundaries. (late 1974) 

texture Separate and nsme wood, metal, cloth, 

paper, plaster, and a few other surfaces. (1975) 

Use texture as an aid in determining 
approximate surface orientation. (1974) 



shading 



description 



Devise a qualitative shading and 
highl ight theory sufficient to 
correctly suggest flat, cylindrical. 
spherical, and conical sections with- 
out recourse to directly measured 
depth information or the numerical 
Integration method. (1975) 

Complete first order theory of tieredi 
hierarchical descriptions and accom- 
panying program for planning toy houses 
from bricks. (Ph.D. thesis In progress) 

Complete a theory of curved object dee- 
cription and accompanying program capable 
of describing vasss in humanly 
acceptable form. (Thesis in progress) 

Extend work In curved object description 
to deal with objects commonly found in a 
kitchen or other complex room. (1374-75) 

Harry description and analysis tools 
into a system able to describe the 
kitchen or other class of objects from 
camera input (1975, but some results already) 



Hater archical 
systems and 
context driven 

ana lysis 



Search for and ident 
ject In a clutter of 
ing dirt, reasonable 

cons i derab I e shape 
(Long-range goal) 



fy a specified ob- 
tools notui thstand- 
obstruction, and 
abberation. 



Hand-eye 
coordinat ion 



Cause tMo objecte to touch gently using 
shadow Information. (1974) 



Devise monitor capab 
monitoring a large 



of efficiently 
1 for motion. 



E, THE riOQULAR VISION AND fiANIPULATION LABORATORY 



International economic problems, environmental questions, worker 
eatiefaction and a host of other issues argue etrongly for the 
development of an advanced productivity technology. Our modular "Mini* 
robot" laboratory effort responds to this need through Ite two primary 
goals: 



1) To develop a modular set of vision and manipulation tool* 
suitable for substantive research that costs less than 
t7S,080. Ue believe that such a laboratory kit will widely 
stimulate research on advanced productivity. Ue Measure our 
success here in proportion to the degree of acceptance of our 
laboratory kit as adopted by other research groups, 

2) To reorient a substantial portion of our oun !aboratory*e 
efforts toward applied work. Ue believe our oun theoretical 
work in vision places us in a strong position for speeding 
toward early application achievements. Progress here can ba 
equated with the use of Machines in industry whose existence 
can be credited to our work. 

Our plan to achieve these goals has three major partsi 11 assembly of 
the hardware by purchase or construction, 2) programming of basic 
software support programs, and 3) successful demonstration of the 

equipment on a specific application task. 

Hardware for the Vlelon anxi Manipulation System 

Hardware development and selection has been a major focus during tho 
first years- Ue now have selected and acquired a computer 
configuration, a vidicon system, and a digitally driven x-y table* A 
new arm designed for us is under construction for December 1373 
delivery, A 512 linear array and mirror drivers are here and the 
deelgn of a new camera using them Is being completed. 

Image Input 

Both the image dissector and the vidicon image sensore suffer from about 
a dozen major defects each. It is reasonabls to build a simple Bn<i 
Inexpensive image Input device using a now-available low-noise, high- 
sensitivity P.I.N, photo-diode, an F.E.T. op-amp and a fast mirror 
deflection system. Interfacing requires only two 0/A*s and one A/D. 
Its speed should be comparable to that of the image dleeectore of older 
vintage running at a comparable signal/noise ratio. The device would 
have lower geometric distortion* scatter, and better uniformity of 
reeponse. (The use of the same sensing device for each point ie an 

1 mpor tan t feature. ) 

In a related effort, we Hill evaluate a system which eliminates one 
dimension of mirror scanning by using a Rettcon linear array and a 
aingle scanning mirror. (These devices are normally used for binary 

Inputs onJy. Us must check out their uniformity, bloom, noise, range, 
and meaningful intensity resolution*) 



Should neither of these "random access" schemes Mork out. our backup 
plan Is to work with an existing image dissector camera head. Our plan 
is to use a mini-computer rather than a special design such ae the video 
processor in recognition of the lou prices noH associated with mini- 
computer processors* 

At the other end of the speed spectrum, vidicon rates are too high for 
any reel time mini-computer processing. One way of lowering the data 
rate and reducing the expense of digitizers and memory space is to read 
only one point per raster line - a whole image nou takes about IB to 15 
seconds* which is compatible with the time It takes to process the 
image. Such a systsm Is expected to be a primary image source during 
the next year. 

Image Output 

Ue do not at the moment havs a satisfactory output device for presenting 
intensity modulated images of acceptable resolution- Such presentation 
ie important both for monitoring how the image input devices perform and 
for presenting the results of processing on the image. The usual 
display devices suffer from a lack of dynamic range* insufficient 
quantization of Intensity, and In some cases from Insufficient 
resolution and flicker. For hard copy, ue can use conventional devices 
with multiple exposure. 

Range Finding 

No entirely satisfactory range-finding method has been developed yet. 
The silt type range-finder using a laser source is adequate for many 
purposes. Time-of-f I ight techniques are under study at the Oraper 
Laboratory, and we feel inclined not to explore this technique oureelvee 
although we continue to follow the Draper Laboratory's progress. 

Ue plan to use random access linear image sensor coupled with a spot 
I ight -source, reducing by a large factor the power required. Suitable 
tracking algorithms should not be hard to develop. Techniques for use 
of visual range information have been explored already by the vleion 
group at Stanford and at SRI, by our associates in Japan, and at General 
Motors, and there Is quite a lot of experience to draw on. 

Manipulation 

More versatile finger configurations have to be worked out- Gripping 
objects which do not have parallel sides requires more complicated 
fingers* possibly pliant and with more degrees of freedom. Ue are 
investigating a multi-finger arrangement which allows the force applied 
to be resolved by strain-gauges built into each finger. 



We continue our interest In trulg small manipulators. Carl Flateau, a 
consultant, is examining scaling laus in connection with a new hand he 
ia designing for us. Another consultant, Russell Seitz, has advised ue 
about scaling problems with physical materials and about further study 
of biological systems on this scale. 

To complement the hand, wrist, and arm work ue plan to develop suitable 
tools and the means to transfer power to then* Programming must 
consider the tools to be extensions of the hand with respect to dynamics 
and force sensing. Tools are nseded fori voltage-probing, cutting 
Mires, soldering, desoldsring, clsaning, holding In fixed orientation, 
bending. 

The Scheinman arm, which is 2/3 human scale, is nearing completion* It 
was designed for us while he was in residence during 1972. Ue may 
contract for construction of Plateau's 1/4 scale arm. These arms will 

facilitate assessment of the usefulness of tachometer feedback, direct 
computer servoing, advantage of counterbalancing and the like. More 
attention can now be turned to wrist and hand considerations. Small 
eemi -conductor strain-guage wrists, preferably of one-piece 
construction, u ' l I be needed. Ue nesd electronics to make it sensitive* 
accurate and drift free. Some work will have to be done on correctly 
resolving the three forces and three moments from the measurements. 
Touch sensors for the fingers will be developed with more spatial 
resolution* Two devices will be particularly investigated. One, seen 

In Japan, ia an array of metallic buttons buried in some elastic 
material. The other involves the use of fiber-optic devices. Light 

traveling down a light pipe Is reflected in proportion to the 
compression of elastic transparent material at its end. Thie promises 

to allow a certain amount of force resolution while being very small at 

the sensing end. Bradford Howland, of Lincoln Lab., Is working on such 
devices. 



Software 

Our overall approach involves splitting the computation requirements 
between the mini-robot*s processor and a larger remote machine with the 
ARPA network serving as the communication medium. The high-level 
knowledge-rich portions of a robot experiment can thus be developed In 
the friendly environment of the large machine with Its greater file 
system and more powerful languages. HeanwhMe the local processor 
handles straightforward programs which ars too data-time and I/O- 
dependent to work well over the ARPA net. 

This gives high priority to the creation of a command interpreter 
capable of interfacing commands from and information toward the larger 
machine. Meyer Billmers is In charge of developing the language. It 
ie to accept commands in the form of character strings consisting of a 



function name followed by a sequence of numerical or symbolic argumente. 
This syntax Is smoothly compatible with LISP and simplifies Interface 
programming at both ends. 

An additional consideration la compatibility with respect to the 
possible development of a powerful POP/11 stand-alone LISP system. If 
euch a system were developed, the mini-robot itself could support all of 
the software development and execution, thereby moving network 
interfacing to the role of program and picture data exchange* 

In many applications. Know lodge-based programs determine access points 
in a small area. Subsequent accesses are likely to be nearby in the 
picture and therefore in core on the eame page. A paging arrangement le 
designed to recognize this kind of use and make relatively few acceeeea 
to the disk. 

Ue therefore plan a software picture dumping system. The system will 
tranefer data from vidlcon and solid state arrays into picture f i lee on 
the local disk. Further interfacing will allow transfer of files to 

bulk etorage on the parent machine and to the ARPA network community at 
large. This will allow low budget groups to work with expensively 

procured images without having the physical image device in house. 

Ue plan to pattern the mini-robot picture facility after that already 
existing on our POP/18 system. Picture arrays will be stored ae 
collectione of eubarraye which in turn can be paged in and out of corm 
memory. 

Experience shows conclusively that disk stored picture files are 
essential for scientific vision research. Uithout such files large 
complex programs become impossible to debug and two programs can never 
be compared with any satisfaction. 

Ue have singled out dynamic arm control as a particularly sensitive area 
In which to begin development of basic user primitives because 
considerable theoretical work must be done before satisfactory control 
programs can be written. 

Some studies have been made in our group and elsewhere (refi Grosser, 
Uhitney, Stanford) but problems remain In that the straightforward 
manipulation of the arm dynamics equations result in solution formulas 
with far too much computation for real time use. Ue expect eventually 
to find efficient symbolic and interpolation/table look-up eolutions to 
such problems. 

Richard Uatere is making good progress toward a set of arm control 
programs and basic commands- He hopes to continue during the next year 
to approach the computational problem with a combination of heurietic 
and mathematical ideas that already has made considerable progress 
toward reducing the real time load from impossible to manageable. 



Demonstration Problem 

Ue believe that a successful piece of hard-core applied research will be 
necessary In order to sell the specific idea of the mini- robot 

laboratory and the general Idea of advanced productivity technology- Ue 
consequently have embarked on the development necessary to enact the 

folloulng scenarioi 

1) A technician specifies an interest In the 
waveform at a particular pin on a given component, 
perhaps a DIP integrated circuit* 

2) The robot locates the part visually. 

3) The robot realizes that the specified pin lies in an 
awkward place. Uires on ths foil side are traced 

to a more accessible place. The arm clips on a test probe. 

4) The technician decides the component is bad* 

5) The robot would then clip off the components 
connec t i ng I ead , 

S) Oesolder the clipped free leads from ths board using a 
force sensitive tug to pull them out* 

7) It uould visually inspsct the holes to be sure they are 
free of solder. 

8) Next, It uould insert the new part, guiding it to the 
correct position uith a combination of visual and 
tacti 1e feedback, then 

9) Solder in the neu part, end finally 

10) Inspect the neuly soldsred joints and resolder if necessary. 

Ue should be able to do this around the end of 1975, probably sooner. 
It should be understood that this demonstration ui 1 1 not be versatile 
enough, houever, to be considered a prototype for a real production 
repair facility. The mechanical activities should proceed at 
approximately human speed, limited primarily by the mechanical hardware 
rather than the computer processing. At that time, ue could decide 
uhether a determined effort should be made to make faster, smaller 
equipment. 

At that point or, perhaps somewhat earlier, ue should assemble a 
conference of people concerned uith delicate assemblies to decide on 



priorities for development of more capable micro-automation equipment* 
Ue will attempt to maintain Mason Hlth agencies involved in such 



concern a* 



Uork Underway 

Timothy Finin and Thomas Lozano have already begun a study of circuit 
board Images. Since a skeleton oysteu ie just noM together, ue have 
begun preliminary studies. Our image dissector ie not entirely 
•atlefactory here because it it both Insensitive In general and 
eueceptible to damage from the highlights that are common with circuit 
boards. Progress, nevertheless, has been made: 

1) Ue now have one good printed circuit wire tracker that 
creates drawings of circuit boards from Images. Three alternate 
algorithms have been blocked out and are being programmed so ae 
to compare their perfomance against the first. Thue some 
progress has been made in scenario problem 3. 

2) Ue have a program that searches for resistors. The program 
has a model for resistor reflection characteristics and Is not 
foolod by other components of the same size and shape. Ue ul 1 1 
couple this program together with those derived from our general 
color studies in order to read color codes. Thie will allow a 
ueer to easily direct the machine's attention to a particular 
resietor. Since our printed circuit wire tracker is running, 
ue can couple in a facility that will Identify components 
connected to previously located resistors. Thie addressee 
scenario problem 2. 

3) Using a prototype force eensitivs arm-gripper combination, 
David Silver demonstrated a force-sensing, -on-visual program 
that turns a crank and spins nuts onto bolts. Ue will 
generalize this to the problem of inserting component Mires into 
holes and the problem of pulling bad parts loose in desolderlng* 
This has preliminary impact on scenario problems 6 and 8. 

The arm control language of Richard Waters will also contribute to all 
activities requiring arm motion. Ths concsntrated work on the elements 
of the scenario will follow the completion of the skeletal system 
assembly with the minimal eye, arm, and software. The entire skeleton 
system should be in useful operation early in 1974. 



PROJECT 2 
THE ELECTRONIC CIRCUIT 0E8UGG8R 

DEFINITION: The goal ia to develop programs that understand the 
principles of ordinary electronic circuits well enough to be able to 
analyse malfunctions in ordinary equipment. Operated in conjunction 
with the PHYSICAL ELECTRONIC ASSISTANT, such systems could perform 
routine maintenance, diagnosis, and repairst They could be equally 
valuable in checking out manufactured products or servicing in the 
field. 

Uhen combined with the physical syetee of Project 1, ue believe that 
this project ui 1 1 lead to a useable maintenance and repair capability 
that* starting in about four years, would demonstrate how to make 
systems that can repair electronic boards, automotive electrical 
systems, and other circuits of similar complexity* 

MILESTONES: 

July 1974i Develop representations for a sample collection of 

us 1 1 -understood circuit "blocks" — amplifiers, detectors, etc.* 
annotated ui th comments concerning electrical functions and 
funct ional roles. 

Construct informal trouble-shooting scenarios, for (say) 
simple transmitter and receiver. 

Dec. 1974: Build formal grammars for the structures in phase 1. 

"Parse" into understandable parts a simple digital logic circuit* 
"Parse - complex schematic diagram of a transmitter into blocks. 
Diagnose and explain why simple faults cause failures- 
Understand design wsll enough to plan signal-tracing 
for particular circuits. 

Select good ssqusncs of tsst~probe points and 
Predict some property of nave-forms at those points. 

Dec. 1975: Extend analyser ("parser") semantics to deal uith detailed 
functional analysis of symbolic circuit diagram. 

Understand design principles well enough to uss debugging theory 
to propose repair. 

Perform physical sequence of signal -tracing test-probe operations 

Connect to Electronic Repairman (Project 1) in demonstration to 
locate (visually) broken part and replace. 

Dec. 1977: Correlate visually-scanned circuit diagram uith physical 
circuit board, to locate components in symbolic diagram. 
Connect to Natural Language system 
Connect to Automatic Programming Assistant 
Repair real circuit uith genuine (field) malfunction. 



APPLICATIONS! The choice of specifically electronice-or lented 
intelligent problem-sol ving is motivated by oor auxiliary concern with 

MICRO-AUTOMATION. In the electronics field* the alternative of using 
human skill to make repairs and modifications will gradually become 

unavailable as assemblies become smaller and more complex. 

(lost work on "robotics" has bsen focussd on initial assembly of devices* 
This Is very natural* because there are fewer complications In working 
Mlth perfect, new components. However, we feel that the most valuable 
applications of Intelligent automata uill be in the areas of maintenance 
and repair, and this area is untouched so far. The visual and logical 
problems are harder, but ue should begin to work on them now so that 
there will not be a very long tine lag when the hand-eye hardware 
becomes adequate for those applications. 

PRO&LEriS: Understanding complex circuits is not a we 1 1 -developed formal 
area. Ue emphasize this because many readers will recognize that 
■circuit theory" has been thoroughly formalized! But circuit theory does 
not *ean circuit-understanding. Ue wl 1 1 first have to develop some 
"scenarios" describing what happens in some simple analog and digital 
circuits. This entails representations of circuit causal functions and 
circuit representations with semantic meaning. To describe the function 
of a component, ue will probably have to talk about "Difference" 
representations for explaining circuit changes. And on top of theee 
high-level functional semantics ue will need a parallel system of 
"physical semantice" for relating the component descriptions to test 
"waveforms". 

In developing such a system, in which one has to work with three or more 
different kinds of representations. Important technical problems are 
shared with those of Project 3 (the Programming Assistant) and point not 
only to the Electronic Repairman application but also to tools for 
perfecting all sorts of large systems. Thsre is no sharp line between 
repair and design. Even In elmple digital circuits, debugging problems 
range from simple broken and shorted connections to subtle symptoms of 
marginal design breakdown — excessive fan-In and fan-out, imperfect 
synchronizing provisions, etc. 

Many problems in this area are new. Of course, previous 'computer 
problem solving" programs had to be debugged, but this was never treated 
systematical ly as a technical problem In itself. Circuits have complex, 
non-serial causalities and depend on intricate "slde-ef fecte" outeide 
the main "signal path". Fortunately, the kind of thinking needed to 
handle such intsractions seem generally similar to the kind needed to 
understand ordinary programs- 
Conventional circuit theory will not occupy center stage. Electronic 
technicians and servicemen do not analyse whole systems as electric 
networks (nor would this be feasible even if they knew the appropriate 
theory). The real problem is to understand the local situation well 



enough to represent it by a simplified circuit, and analyse that, 

COSTS: The project usee the equipment of Project 1 for physical 

hardware, and ue are already funded for that- However the project 
has major computational costs as well. 

PERSONNEL! G. Sustean, I, Goldstein, Scott Fahlnan. 
Ulth n. Mlnsky, S. Papert, P. Ulneton, 
C, Reeve, T. Knight, J. Hoi li 



TECHNICAL ASPECTS OF PROJECT 2 

THE ELECTRONIC CIRCUIT DEBUGGER 



A. INTRODUCTION 



DEFINITION; The goal is to develop a system that understand* the 
principles of ordinary electronic circuits well enough to be able to 
analyze malfunctions In ordinary equipment. Ue uant It to be applicable 
to a wide variety of devices. Also, one would like to be able to extend 
It to new classes of problems* wl thout too much effort- Therefore the 
system must be based on good principles of causal and teleologieal 
reasoning. All special knowledge of devices and componente should be 
nodular and extensible. 

It ie not necessary to nake the "learning" as easy as It 1e for a human 
technician — to make the system have practical value. An advantage of 
a computer -based eystem Ie that once the skill is learned it can be 
transferred swiftly to other machines* 

Or so i t would seem! Uhat computer scientists have learned, however, ie 
that one can rarely "add" two skills together to make one larger, better 
program! One might even say that a basic problem in A.I. is td find 
representations for skills in which the interactlone due to such merging 
can be easily Isolated and wade compatible. 

In order to avoid escape into toy problems, we intend to develop thie 
eystem in several parallel domains of real devices. Eventually these 
uii i include analog devices ranging from consumer radio, T*V* t and hi- 
fi through sophisticated communications equipment and digital devicee 
ranging from the pocket calculator through the mini-computer. The 
underlying reasoning processes should eventually be knowledgeable enough 
to cover thie range and be quickly adaptable to new kinde of componente 
(provided the basic circuit concepts are not changed much), 

Ue expect thie kind of eystem to be useful In augmenting the 
intellectual powers of (human) maintenance technicians. Uhen combined 
with the Physical Electronics Repairman (Project 1) it could perforin 
routine maintainanee, diagnosis, and repairs by itself. At what level 
of skill? It ie hard to say at this time. Clearly, it will be very hard 
to approach the general common sense of a skilled technician* On the 
other side, the machine should be able to exploit special kinde of 
expertise in using conventional circuit theory, correlation of 
simultaneous measurements, etc. Of course, we expect all this research 
to work directly toward improving our position vis-a-vis general 
knowledge, so perhaps this is too conservative a position. 



B. GOALS 

PROBLEM. Suppose that mo are given an inoperative electronic device of 
known correct design- Deduce, from its symptoms and from a feu well 
choeen experimente, the cause of difficulty. (You have to choose the 
experiments, first* by understanding the symptoms!) Determine the action 
to be taken (e.g.. the components to be replaced) to restore the device 
to working order. 

Why le thie a good problem? 

SCIENTIFIC VALUE: It is particularly important to etudy qualitative 
causal and teleological reasoning. Thie ie one of the weakest areas of 
AI. This task requiree sensible and purposeful experimental and 
exploratory behavior. It attacks head-on the problem of dealing with 
the unexpected In real world situations. 

ECONOMIC VALUEi There is a lack of highly trained technicians to 
maintain modern complex electronic hardware. If coupled with a robot In 
the future it could be valuable for repairs In hoetlle environments 
{space, undersea etc.). Uith quickly evolving equipment* ae In siodern 
electronics, the problems are unusually severe. 

The problem also engages another Important modarn focus in our work. 
Tha repair problem contrasts with the harder kinds of design and puzzle 
problems because 

> In repair, one ie usually given a description of how thinge are 
eupposed to be; how the device should work. One doesn* t have to 
figure that out. Nonetheless, one has to understand the explanation! 
So the problem meete the condition that before one can create a 
"designer 11 or program-writer, one should know how to build a 
repairman. I.e., an understandsr * annotator - debugger. 

> The program needs less knowledge, in the sense that one can 
understand how a device works without knowing all the deelgn 
considerations of how to Invent It* 

> The apparent solution to the problem seems to fit in well with our 
current concept of the "fra»e B or "scenarios-type recognition and 
exp I anat i on theory. 

Thus, we need only determlnehy the device pressnted doee not operate 
according to en understood model of Ite operation* 



C. A tlOOEL-DfllVEN TROUBLE-SHOOTING SCENARIO 

Imagine an AH superheterodyne radio receiver — say one of the standard 
5-tube AC-DC circuits that Mas once the most common of all electronic 
devices — with the symptom that loud signals are distorted. Suppose 

further that the problem Is in fact caused by a shorted AVC filter 
capacitor. How does our repairman determine the cause of failure? 

Uhen the problem is posed, our repairman pulls out a fairly abstract 
model of a superheterodyne radio receiver (See Figure 1. How our 
repairman assimilated the model is discussed elsewhere). This "general" 
model actually covers a large class of radio receivers. Including most 
traneietor portables as well as the "al l-Amsrican 5". (It does not, 
however, cover basically different designs, e.g., the simpler Tuned - 
Radio - Frequency receiver.) Of course, this Is not the only possible 
model for a euperhett one might conceivably find a quite different way 
to analyse it into modules. In any case, this model does distinguish 
submodules, each of which can be further specified as a module with 
specific ports and some internal structure. Ue see that the model also 
distinguishes the basic signal path from control signal paths and power 
paths* Each eubmodule is likewise specified. Ue show two possible 
converter modules in Figures 2 and 3. • The first one is appropriate to 
most broadcast receivers, but the sscond might be found In fancier 
communications equipment. 

There are analogies both ulth physical scene-analysl s and with a 

■generative grammar" of electronic equipment. The top level of grammar 
-- like the "kernel sentences'* — are the devices that people use* The 

lowest level ~ the "words" or "terminal symbols" — are the atomic 
components; resi a tor, capacl tor, inductor, transistor, etc. Thl e 
grammar Is matched against the schematic diagrae of the equipment to be 
debugged, to establish the correspondence of parts of the model to parte 

in the diagram. The result of this "parse" is a hierarchically 
annotated diagram, ulth the module boundaries laid out and each module 

(down to the atomic componsnts, if necessary) annotated with Its 
purpose. 

It will be Interesting to see whether we can usefully model euch 
analyses within the proposed ACTOR system Isee Project 31 and obtain an 
analogy to conventional signal-space "simulation". In the Actor 
application, the circuit blocks might converse by means of messages 
describing the "wave-forms"! In conventional simulation, no one has 
gone outside the basic time-signal space. Each component module (at all 
levels: amplifier to resistor) can be described extrinsical I y and 
Intrinsically, Intrinsic descriptions — that Is. looking down the tree 
— describe what the module is: examples aret 
6.1 ufd. capaci tor 
2 MegOhm resistor 
45S KHz tuned circuit 
Fixed-tuned radio- frequency amplifier. 
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Communications Receiver Converter 
(power connections ignored for clarity) 



An extrinsic description explain* the use or purpose of that module In 
the overall circuit — in terms of higher-level nodes of the tree* 
Corresponding extrinsic descriptions of the earn objecte arei 

cathode bypass capacitor 
grid- leak resistor 
IF amplifier Input tank 
IF amp I if ier. 

Usually* there Is only one intrinsic description of a particular pile of 
parts but the extrinsic descriptions depend upon context — their 
relation to each other. 

A device ie broken if the behavior of that device is not as advertieed 
In its intrinsic description. Since the devices us uill deal with arm 

correctly designed this Means that some subcomponents do not live up to 
their Intrinsic descriptions. The problem Is then to determine which 

atomic components are at fault (uithout testing them all individually). 

Let us return to the given problem and consider the reasoning of our 
repairman. The perception of distorted sound on loud signals means that 
some module in the basic signal path Is not linear with respect to the 
audio component of the signal. Uhere is the distortion introduced? The 
repairman program instructs us to set up the radio on the workbench 
under operating conditions. The radio is tuned to the output of an 
amplitude-modulated signal generator Mhich l» adjusted to product * 
signal strong enough to cause the problem. The program now traces back 
the signal path looking for the place Mhere the distortion occurs. It 
asks us to "scope" the audio output and determine if the distortion ie 
there. Ue answer yes. It then asks about the output of the detector. 
The ansuer is still yes. In fact, ue find that the distortion 
originates in the IF amplifier because its output is distorted but ite 
Input ie not. Is the problem in the IF amplifier? Let's look at the 
other inputs (prerequisites) to the IF to check if they are reasonable. 
The power supplied Is OK but ue find that the AVC line ie at B Volte and 
that it is independent of the RF input (from the signal generator). 
Something is wrong, then, with the AVC bus. By considering the 
consequences of the AVC being held at 8 Volts we can see that thie could 
cause the problem* The AVC controls the gain of the converter and the 
IF amp. At Volts everything Is at maximum gain. A strong signal 
would then be amplified enough to drive the IF amplifier Into non-linear 
operation, hence the distortion. 

Now, what is the problem with the AVC bus? Is the problem that the AVC 
voltage ie not being generated at the detector, or Is it being bypassed 
to ground in the converter or IF amplifier? The uay to determine thie 
ie to disconnect the AVC line from the converter and IF amp and then 
measure ite voltage when Isolated. Still zerol Thus the voltage Is not 
being generated. Let's look at tho dstsctor in mors data! I (Figure 4. 



Figure 4 matches Figure 5 in the actual circuit). Since we get an audio 
signal out of the audio filter (tilth a OC component) the problem must be 
In the OC filter. But that leaves only two components to test* the 3.3 
Megohm resistor and the .lufd capacitor. Ue find that the capacitor le 
shorted, hence the problem le eolved! 

In the preceding scenario of the operation of a competent repairman 
mc* get a glimpse of the general approach. The device ie broken if It 
does not behave as advertised bg Its intrinsic description. It may not 
provide the expected output for a specific input. The question Is, "How 
should the correct output be generated?" Ue then look at the structure 
of the device, as dsscribsd bg the parse tree to determine what 
eubmodulee are directly responsible for generating the output. In an AM 
radio this Is the audio amplifier. (Recursing down, if the device Is an 
audio amplifier, its main tisp s the output stage. If ths output stags 
Is push-pull then there are multiple main stsps — components which 
contribute directly to ths generation of output of the next higher level 
module,) If the device is correctly designed (as ue are assuming) either 
one of these main steps le incorrectly operating or It Is getting bad 
Inputs. Ue use this idsa to tracs back along ths main signal path for 
the first place where the signal appears good. The problem is then 
either in this stage, the auxiliary inputs to this stage, or the 
Interface to the next stage (the output of the current stage may be 
overloaded), Ue check each of the possibilities and then, uhen ue have 
found an inoperative submodule, Me recursively apply this analyeie until 
the bad atomic components are isolated. 

This ie of course a sketchy idea and It must be refined. How does it 
relate to other kinds of troubleshooting, like ths debugging of computer 
programs? Can these Ideas be extended to debugging of design errors ae 
Hell as component failures? The answers to thsss questions are basic 
research goals. 

0. MILESTONES 

In attacking a problem such as this, it is important to thoroughly 
understand a variety of instances bsfors designing a "general" method. 
Thus an important first step is to work out detailed scenarios (far store 
detailed than the one shown hsre) for a number of electronic 
troubleshooting tasks in devices ranging from transistor portable radios 
to test equipment. Hucb can be learned by discussion with competent 
technicians as they perform maintenance in the AI lab. As this evolves, 
we have to formulate representations of ways to "explain" how the 
circuits work — as seen by repair technicians. Thie ie very different 
from the electric network theories learned in academic electrical 
engineering courses; it is qualitative common senss. 



PROJECT Z ELECTRONIC CIRCUIT DIAGNOSIS 
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After compiling some sot of scsnarios, the next step is to generalize 
the results. At this point. It sould begin to be clear just what arm 
the general techniques of electronic troubleshooting. Ue now begin to 
design a program which can parse a schematic diagram with respect to an 
electronics grammar assigning to each part in the device its purpose In 
the circuit. 

By the end of the first year ue should have a comprehensive grammar 
covering some small class of electronic devices such as commonly 
available consumer radio receivers. This would include, of course, many 
of the building blocks of more complex equipment; we would have quite a 
zoo of oscillators, amplifiers, etc., which are also found in televieion 
eete, radars, and communications equipment. 

Shortly after the first year we should also have a program capable of 
parsing a schematic diagram with respect to this grammar and answering 
questons about the result, such ast 

I. Uhat Rind of local oscillator is used in the converter of thie 
device? Answer! A Hartley Oscillator, 

2* Uhat is the purpose of C18 In this device? 
Answers It is an emitter bypass capacitor. 

By the end of the second year, ue should actually have a program running 
which can use the information provided by the program written in the 
first year to perform aimple troubleshooting taoKe ainilar to the onee 
collected in the scenarios. Ue expect that by this time many scenarios 
will have been developed. During this second year, we expect to 
Increase the size of our grammar to cover many new device typed such as 
transmi tters. 

Ue wish we could etate at this time how hard it wilt be to add knowledge 
about new circuits to the system. But this is a new kind of problem* 
and no one has had any experience with the pseudo-lingui stic problem of 
meaningful circuit grammars. Uithin a uell-defined category such as 
communication receivers and transmitters, we would not expect much 
difficulty in passing from one model to another — except that each 
"set* is likely to contain some designer's favorite idiosyncracy, 
requiring special attention. Designers are prone to obtaining a bias 
voltage from some unusually stable side-effect. They will insist on 
using some device's Internal resistance as a feedback common element* 
These conflict with generally followed design heuristics and can cause 
trouble in diagnosis. In any case, by the end of the first year we 
ehould be able to assess the potential difficulties. 



PROJECT 3 
AUTOMATIC PROGRAmiNG, DEBUGGING, and DOCUMENTATION 

DEFINITION: Ue Mill attempt to make systems to facilitate development 
of complex computer program*. The goals Involve automatic analysis, 
automatic documentation and debugging schemes. Because the performance 
will be defined by exterior goals, these systems ulll have to Include 
advanced learning abilities. 

There are really tMO projects here with two approaches; one Is concerned 
with representing commongense reasoning knowledge, as exemplified by the 
theeee of Sussman and Goldstein, Project 3. If the other is concerned 
uith developing a programming formalism and technology, the ACTOR system 
of Hewitt and his associates. In which implicit and undesirable 
interactions arm (we hope) unlikely to arise accidentally — Project 
3.2. 

MILESTONES! - 

July 1974; Formulation of intention formalises in several microworldei 
Blocks world debugglngi extensions of Suseman's Thesis 
Semantics of graphics: extensions of Goldstein's Thmele 
Semantics of electric circuit simulation programs 

Dec. 1974: Preliminary ACTOR formalism realization 

First attempts to apply automatic dsbugging methods to 
electronic circuit problems, 

Dec, 1975i First Applications to Electronics Diagnosis and 

Repair programs (Note — programs, not problems!) 

First Natural Language Interfaces 

Programming Assistant for the Personal Management Assistant* 

if that develops into an AI Lab project. 

APPLICATIONS: Work on Artificial Intelligence, over the past few years, 
has created an environment in which we can ask much more from computer 
programs than was previously reasonable. At least, thle Is so "In 
theory". But the much more complex programs required for this are 
harder to understand, modify, and debug. Thus, this progress ulll not 
pay off as well as It should until there are corresponding tmprovemente 

in: 

Responsibility -- Ability of the prograns themselves to Juetlfy their 
results! to explain what was done to get the reeult. 

Accountability — Ability to explain the assumptions the reeulte are 
based on. 



Debuggabi I i ty — Prograna can provide much bettor information to make It 
easier to detect programming mistakes. 

Extendabi I i ty — Programs should have enough nodularity and 

"transparency** to details to make extensions possible without 
requiring the programmer to understand all the fine details* 

Ue believe that it is possible to make major advances in these areas 
now, because of recently developing capabilities to incorporate into 
programs a new kind of knowledge. To be epeciflc, we mean the KNOULEDGE 
ABOUT HOU THE PROGRAM UORKS — in addition to what programs 
traditionally contain* procedures designed specifically to solve the 
target problems. * 

PROBLEMS: In our approach, the main directions are already outlined in 
the prototypes presented in the recent theses of Sussman, 
Goldstein, and in the Actor-Intention paper of Hewitt et at. These 
all appear to give rather definite and rather promising outlinee of 
what must be done. As the different kinds of principles in each of 
these are clarified and applied to different microuorlds, we can 
expect many difficulties, interactions, and conflicting 
priorities. It Is too early to guess which problems will be most 

is of concern to two distinct groups. 
C. Hewitt 
P. Bishop 
et al. 

The results so far have captured the imagination of many studente* 
and ue expect that quite a few more people will become involved In 
this project. 

COSTS: This "project" requires large computational support. It aleo 
involves resources of the sort found in general computer language 
development, systems programmsrs and maintenance people* 
documentation, etc. 

The systeme will use features related to LISP, MICRO-PLANNER, 
PLANNER, CONNIVER, LLOGO, and the new ACTOR eystetn. 
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PROJECT 3.1 
PRINCIPLES OF REPAIR AND DEBUGGING 



A, INTRODUCTION 



Uhat makes an Individual a competent repairman? Is there a set of 
skills which are common to the sxpsrt television repairman, computer 
programmer or digital logic troubleshooter? Ue assert that there 1e 

indeed an Important core of knowledge common to these activities. It 
consists of expertise in debugging, planning, skill acquisition and 
modular knowledge representation. Ue propose to dsvelop this area by 
building ''repairmen" for the above domains. Ue hope to do thie In ways 

that both produce "expert" programs for each area as well as reveal 
sound principles applicable to other domains. 

The Conceptual Framework 

To design a programming assistant or a circuit repairman is useful In 
Its oun right. The project takes on further interest if one believes 
that there sre important skills and knowledge that ars "generally" 
useful in repairing things. For, if such knowledge could be accumulated 
in a heuristic program, the design or adaptation of repair-programs for 
different domains would become progressively easier. 

Readers who followed closely the development of AI in its early gears — 
but oot in the last 3 or 4 — might be inclined to sayi "That sounds 

like the early schemes of separating knowledge about problem-solving In 
general from knowledge about particular areas of expertize. Didn't It 

turn out that this line led to mediocre results? And doesn't it turn 
away from the principles of Ksterarchy that you have been advocating. In 
which different kinds of knowledge have to work together?" 

For the last question, ths reply Is simply that the kinds of knowledge 
do indeed have to uork closely together. But ue claim that In the 
earlier "general" problem solvere inadequate attention was given to 
questions of repair and debugging — and that la really why they turned 
out to be relatively weak) 

Thie debugging knowledge is precisely what Is needed to close the gap 
between planning and execution. Some of the early programs did indeed 
have some "general" knowledge about hou to solve hard problems by 
breaking them down into simplsr sub-problems. But then they faltered. 
For It Is in the nature of a "general" plan that it does not take 
details Into account. Most plans fall down, in fact, when It comes to 
details* The only hope is to REPAIR the most plausible plan. And this 
Is what we propose to study systematically. 



B. THEORY OF DEBUGGING 

Debugging is an essential component of reasoning. Plane rarely work the 
first tine; and even when they do, the uorld nay change. Furthermore, 
debugging is an essential component of self-improvement. Ue often learn 
by debugging our own knowledge. One can regard Ulnston's learning 
program (AI TR-76) as an error-diagnosis error-correction debugging 
system. 

One night at first believe that the study of planning and debugging Is 
by its nature informal — mere common sense. In our recent work, 
however, this area has made progress touarde becoming a systematic 
theoretical subject. Perhaps the best developed illustration of uhat 
has happened Is exhibited In the recent thesis of Sussman (soon 
published as AI TR-292) t In which we see the start of an effective 
classification of types of bugs, how to detect them, and how to make 
plans to repair thee. The interested reader should consult the thesis* 
Mhich should be available by the time this proposal Is in circulation. 

The following discussion highlights important concepts of this embryonic 
debugging theory. To illustrate the ideas* we use an example drawn fro* 
the thesis of Goldstein (soon published as AI TR-234) which Is nearly 
completed: it it from the world of 'turtle* programs. 

A "turtle" program is a sequence of commands, to a mobile robot, that 
cauee it to draw a graphic figure on the floor. Alternatively, the 
command* can refer tQ • display simulation Hhsrein the turtle behaves 
like a pen. In the graphics programming language, one can tell the 
"turtle" to go forward or back CFORUARD G0 rt or ""BACK 108*) or to turn 
about its center through some angle ("RIGHT 98" or "LEFT 3S9"). The 
turtle primitives are embedded in a high level symbolic language capable 
of Interpretive evaluation, recursion and iteration. For the purposes 
of this discussion, the programs are given in LOGO syntax, Thle is for 
clarity! the programs could equally ucU be expressed in LISP. 

Thie world of turtle graphics provides a particularly clear distinction 
between imperative method (local movemente of the turtle) and 
declarative intent (static description of the final picture). However, 
examples of powerful planning and debugging could just as easily have 
been drawn from the robot's block world. 

Goldstein's thesis Is concerned with repairing turtle programs which 
fail to draw an extended picture. An example of a typical problem le 
shown in the following wayt 




Figure G. DEBUGGING FACEttAN 



To gain insight Into the process by which the suetee debugs programs, we 
shall draw on the fol lowing siapler procedure for a stick flgurai 



TO STICKflAN 

18 VEE 

28 FORUARO 188 

3B VEE 

48 FORUARO 106 

58 RIGHT 98 

68 CIRCLE 

END 



TO VEE 
IB RIGHT 45 
28 BACK 109 
38 FORUARO IBB 
48 LEFT 38 
SB BACK 188 
GB FORUARO 1B8 
END 



This progra* Mas Intended to draw the following, picture 




Figure 7. THE INTENOEO PICTURE 
But the program has a bug. It actually drawai 



starting heading - 



- 278 degrees 




Figure 8. THE BUGGED PICTURE 



Debugging require* descriptions of Intentions 

To repair the program, a debugging system must initially be provided 
tilth a model of the program** intent* Such a model ie necessary if the 
system ie even to recognize that the program uas unsuccessful. 



For our atick figure, it will be sufficient for the "model 
of geometric predicates describing the desired picture- 



to be a aet 



MODEL STICKMAN 

Ml PARTS HEAD BOOY ARHS LEGS 
M2 CIRCLE HEAD 
f13 LINE BOOY 
114 VEE ARftS 
ri5 VEE LEGS 

MG CONNECTEO HEAD BOOY, CONNECTED BODY ARMS, CONNECTED BOOY LEGS 



ENO 



H7 BELOU LEGS ARflS 
ri8 BELOU ARflS HEAD 



Thla model is underspecl f led. But It tel It 
recognize that the program fails to accompf 
ie to imagine a program, called INTERPRET. 
Hi th the models 



enough of the etory to 
Ish Ite task. Now the reader 
that comparee the program 



(INTERPRET STICKMAN) 

;The eystem ie asked to interpret the picture 
idrann by the program In terms of the model- 



(^VIOLATIONS ithe picture falls to satisfy the model because* 

(H5 (NOT (LINE BODY))) [The body la not a Una. 

(H7 (NOT (BELOU LEGS ARMS))) |Tha legs are not below the arme. 

(MS (NOT (BELOU AMIS HEAD))))) |The arms are not belou the head. 

Different domains Mill require different model languages. But It wli I 
always be necessary to specify the purposes of major parte and the 
relationships between these parts. 

Deacrlptlone of Programs* Intentions: Annotations 

Annotation is the process of building a detailed description of 
intention and effect. The "intentions" describe the "why" or reasons 
for the object's etructuret the "effects" describe a causal chain 
documenting its performance. Part of the explanation is specific to the 
particular domain. But important constituents 9t9 unlvereal. Thie 
would include structuring into main steps and state interfaces, 
aesigning responsibility for parts of the task to parte of the mechanism 
and documenting Interactions accidental to the main purpose. 

The "Plan" — * the top level of Annotation 

Deriving tha plan auppllta tht "why" of tha ayttem, whether 
program or circuit. The plan describes hou the goals of the model aro 
to be accomplished. It la an aasignmsnt of local reeponelbi 1 1 ty between 
statements in the model and modules or interfacaa in tha code. Deducing 
the plan is based on general knowledge that processes consist of main 
steps Interleaved with setups* Interfacee, and cleanups. 

The Idea of "state" 

Of course, ANNOTATION requires understanding domain-dependent, non- 
universal knowledge also. For turtle-program graphics, cognizance of 
the STATE - position and heading - Is Important. For clrcuite, the 
state will be In terms of electrical primitives — It might be an 
instantaneous list of all currents and voltages. But In both cases, 
analysis proceeds in determining the purpose and success of each stage 
In 4 terms of its effect on the state. 

Our point is the modest one; that there may also be "general" principles 
about how to handle particular problems. The idea of defining a "etate" 
might be critical for euccess in many sreaa, even though each requires 
different kinds of states, with different kinds of transformation 
properties. 



Rational Design and First-Order Theories 

Here Is another very general piece of knowledge (or "advice"). Finding 
a plan ie simplified by assuming that the system under study Mae 
designed according to the following "rational design" principle. 

The modules interact only over explicit interfaces. There are 
no accidental aide effecte. This is the "Firet-Qrder" theory; 
It le surely falee in some way, but it can guide the initial 
attempt at underetanding. 

Later, ae difficulties are isolated, the bug nay indeed be discovered ae 
due to a violation of "first-order" principles such as an unexpected 
Interaction between supposedly independent sub*syetems. 

Urn cannot give details hsre, but a psttern~matchlng procedure based on 
thie principle yields the fo Mowing plan for the STICKMAN programi 

TO STICKMAN 

IB VEE (ACC0T1PL1SH LEGS) 

28 FORUARO 198 (ACCOMPLISH IPART1 BOOY>) 

3B VEE (INSERT ARMS BODY) 

4B FORWARD 198 (ACCOMPLISH (PART2 BOOV)) 

58 RIGHT 9B (SETUP (STATE HEADING) (FDR HEAD)) 

68 CIRCLE (ACCOMPLISH HEAD) 

END 

NOTE: This plan ie the RESULT of the running of a phase of 
Goldstein's program, Thus, It might be considered to be an 
Automatic Program Annotator. In the programs of Sussman, 
"comments" Bre provided ab Initio to the programs to be 
debuggedt in the courae of operation more comments sre 
generated. In Hewitt's proposal (Project 3.2) the actor 
implementation is enviaioned to detect incompleteness of the 
intention structures, and interrogate the programmer- Thue, 
these are all different threads of the same weave — all 
trying to formalize the relations between intentions and the 
programs written to achieve them. 

More on Plans 

The plan for STICKMAN is almost linear. The legs, body and head 
are accomplished one after the next in a natural sequence appropriate to 
their rotative position- However, note that the purpose comment for the 
arms declares an "insertion". This is a common and useful type of 
abetract planning etructure to which the system is sensitive. 



Uhile accomplishing one part of a model, the program may be in 
the appropriate entry state for another part. In this case. It is 
natural to "accomplish" the arme In the midet of drawing the body- If 
the program for the Inserted part Is etate transparent, then the system 
can expect that the intrusion will cause no harm. Of course, the 
Inserted procedure may interact In unexpected Mays with the main program 
or eimply not be state transparent. However, by being sensitive to this 
abstract plan format, the system Is In a position to recognize euch buge 
and fix them accordingly. 

C. DEBUGGING 

Isolating a bug is accomplished by finding inconsistencies 
between intention and effect. Debugging is accomplished by deecrlbing 
the type of discrepancy and making the appropriate patch. 

Commom Underlying Causes 

The underlying causs of the disaster can often be described with 
sufficient abstractness to apply to many domains. For example, such 
causes as CONFLICTING-BROTHER-GOALS, UNEXPECTED-SIDE-EFFECT, riOOULE- 
FAILURE and I HPROPER- INTERFACE are universal causes of failure. <NOTE> 
theee terms mean pretty much what they seem to mean. For details, they 
are defined precisely in ths Goldstein and Sussman theses.) 

Coanon Hsthods of Rspair 

Similarly, there are Important common elements in strategies for 
repairing bugs. CONFLICTING-BROTHER-GOALS can sonetimee be fixed simply 
by reordering. Interface problems are simplified by maximizing etate 

transparency. HOOULE -FAILURE, whether of a sub-procedure, tube or chip, 
suggests the obvious correction of recurslng the system and repairing 

(or replacing) the module* 

Ordering the Attack 

flultiple buge can be difficult to correct. Hence, guidelines 
are necessary in the order of debugging. A heuristic of wide 
applicability is to debug the parts bsfors attempting to correct th# 
relatione between them. For the STICKMAN, this would mean debugging the 
body before worrying about the "above" relations. The basis of this 
ordering Is the standard scientific notion of beginning with a flrat- 
order linear theory of a problem before attempting a eecond-order 
explanation uhich handles interactions. 



The plan indicatee uhich steps arm responsible for the body. Oomain 
dependent knowledge define* a lint at composed of col i near vectors, 
where FORUARO* s are understood to produce vectors* "Co linear* is a 
constraint on the direction of vectors. Now, dovaln Independent 
knowledge Is applied. An abstract pattern-match of the procese it made 
to diecover the etate-interfaces between the main steps responsible for 
the body. In this case, the Interface Is line 38 wherein the arms ar^ 
drawn. The bug Is classified at an UNEXPECTED-S IDE-EFFECT of VEE, The 
fix ie to Insert a patch returning the turtle to the correct headlngi 

INSERT LINE 35 "RIGHT 45" 



Aeethet re Interrupt 

An aesthetic interrupt occurs when the criteria of good design are 
violated. These criteria ar^ based on considerations of efficiency, 
clarity and resistance to future bugs* 

In thie exanple, the sub-procedure VEE already 
returns the turtle to its entry position. Inserting 
the interface in line 35 of STICKttAN requires that 
"heading" also be restored. The result ie 3n 
abstract pattern natch on procedures in which 
"state-transparency" Is required. The result of 
this match ie to rmmovm the "RIGHT 45" from STICKrtAN 
and, inetead, insert it as the final line of VEE. 
This restores the original heading and VEE becomes 
fully transparent with respect to both position and 
direction. 

INSERT LINE 65 OF VEE "RIGHT 45" 

State-transparency la en important characteristic for achieving 
modularity. Thus, thie edit serves to make the VEE sub-procedure 
simpler to use in future applications. 



Recursion 

Having fixed the "body", the system now recurees and debugs the 
remaining difficulties, Uith VEE edited, the STICKTMN now has the 
appearance: 




The failure of the above relations can be classified as having one of 
two possible underlying causes* The first is that the interfaces 
between the parts are in error. The second is that the global etate 
upon entry ia not as expected. Under the former assumption, each 
Interface must be debugged* Under the latter, only one change need be 
Mda. 



n in. ma I Change 

A repairman should sake Minimal changes to the system. Ite goal le to 
fix the eyetem, not redesign It, (lore Important, It doea not fully knew 
the deslgner'e Intent, Hence, It should be hesitant to make major 
revielone in hie plan. Thus, a single edit to the entry Interface la 
clearly preferable to many edits to Internal Interfaces. 

The reeulting change to 5TICKI1AN 1st 

INSERT LINE 5 OF STICKMAN "LEFT 98* 

EXPECTATION IENTRY STICKHAN) (HEADING - 278) 



Expectations 

An important aide effect of this edit Is the insertion of an 
expectation. The expectation checks the entry state to STItXrtAN. In 
the event that It is not 90 dsgrsss, the system is immediately cognizant 
of an anomaly* It la not necessary for it to repeat again the entire 
analyele It flret performed. It learne from experience by adding 
commentary to the user*s code, 

0. KNOWLEDGE -- SPECIAL ANO GENERAL 

Specialized knowledge etructures for electronic circuitry* programming, 
and digital logic will be required to Interact Mith the basic repairman 

module. This Mill force the system to deal with baelc lesues in the 
management and use of targe collections of knowledge- Several recent 

ideas in Artificial Intelligence make this difficult teak seem 



manageable. 



Demons 



The first is the use of demone. Old-fashioned programs required an 
explicit control structure, dispatching In sequence to a prepared Met 
of sub-procedures. This becomss inadequate when the number of 
situations that the system may encounter grows large. Demons Brm 
programs that are automatically activated when a datum or request 
matching their pattern becomes current. 

Procedural Knowledge 

A eecond tool Is the representation of knowledge as procedures. Repair 
know-how is not a collection of facts but a set of dirsctlons. A 
uniform reasoning program is too inefficient* It does not exhibit skill 
or expertise. Demons can be procedures of arbitrary complexity. 

Virtual Databases 

Some knowledge is most clearly thought of as factsi for example, the 
etate of a circuit or computer process. This may. however, be too 
burdensome In terms of space. The solution is the use of a "virtual* 1 
data base, flost of the state is not of intersst and ii, hence, not 
ordinarily computed. The repairman, however, never knowe this* Upon 
requeet, invisible demons compute and asssrt the nseded data. Hence, 
space la used only when nesdsd but rsdundant computation is avoided. 
This management system provides important conceptual clarity. 

For turtle programs, this structure is used for asking geometric 
questions of the picture. The asker expects to find the answsr In the 
data base. It may well be there explicitly as a result of analyzing the 
performance of the program in careful mode. Thie would Include 
etatements of connectivity between sequentially drawn vectors. However, 
the answer may not be present. A "global" connection due to the turtle 
crossing a previously drawn vector re not easy to notice while running 
the program. Upon asking for such Information, however, demons Bra 
activated which compute the answer and place It In the data baee. 

The same structure is found In BLOCKS-WORLD programs (see FmhlMn'a 
theeie, AI TR-283) where it is even more time consuming to recompute 
three-dimensional geometric predlcatee. 



Hierarchy 

Representing C0WL1CTING-BR0THER-G0ALS and its associated patches 
abstractly alloue the sane knowledge to be applied to many domaine. 
Programs or circuits In conflict for the same resources can be handled 
by similar plans. Hence* an Important epi steaological goal of this 
research is to represent Knowledge hierarchically In Increasing levels 
of abstraction. 

NOTE: This does not conflict with the notion of a heterarchical 
analysis. Knowledge at different levels eust communicate flexibly* A 
linear flow of control is not adequate. 

The belief that baaic debugging, planning and learning skills can be 
abstracted to a general but powerful fore Is encouraged by the eucceee 
with which the LOGO project has taught such general skills to children. 

\ 

E. LEARNING 

The design of a repairman would not be satisfactory without a non- 
trivial learning component. This is required not only by the desire to 
see the system generally applicable to new domains but also to 
facilitate ite development even for the specific mini-worlds chosen. 
Large systems that sxhibtt no learning are extremely burdeneome to 
program. It ie difficult to predict which lines of research will 
contribute most to this goal. However, here are several promising ideas 
under deve I opment. 

Declarative Programming — 'Building Programs From Advice" 

Declarative programming is an important type of learning which no 
systems currently exhibit. This Is the specification of the task or 
edit via simple declarative advice. The system is responsible for 
making the appropriate compilation or modification of its own code. 

This style of programming is important for several reasons. For one. It 
allows the systsm to exerclss (and the creatore to Judge) Its planning 
and debugging skills. For another. It is easier for the system to debug 
Iteelf, eince in expanding declaratives Into code, the eyetem can fully 
comment the intended purposes. But the most ssssntlal reason Is that 
without such capability, only a person familiar with the entire eyetem 
could possibly make any improvements. 

An essential ingredient to support such capability Is debugging skill. 
The firet expaneion of declaratives Into cods may well have bugs, All 
of the contingencies may not havs been considered. Declaratives do not 



specify interactions. But if the system Is capable of debugging, then 
unforeseen difficulties can be fixed when encountered* 

[n the above STICKtIAN example, declarative programming ie illustrated bg 
the fashion in which the syeten fills in the details. In the form of a 
*plan", to supplement the program- Independent specification of the 
model. Indeed, it is clear that such a systea, with minor extensions, 
je capable of writing turtle programa given only an Initial model of 
Intent, 



Advice Taking 

Even if the ayatem ia unable to debug itself, it understands a language, 
i.e. a eet of concepts, In which it can be given advice about i te own 
processes. This set of concepts Is simply the Ideas about control, 
planning, typee of buga, and methods of solution that it must know 
anyway to debug programs. Thus, if the turtle monitor is unable to 
discover the plan fro* the model and program, it can ask the user for 
etatements of purpose. The concept of "purpose" is part of its 
understanding* 

Skill Acquisition— Improving from Example 

Improving from examples ia a eecond important characteristic. Such 
improvement can be categorized aa debugging oneself In the light of new 
knowledge. Again, this capability Is important If the repairman la to 
be readily extendable to new domains* 






Dead-End Analysis 



A common element to recent work In Artificial Intelligence Is the 
ability of a eystem to learn fro« errors. The metaphor of a eearch for 
the right path Is replaced by an exploration, which ie sensitive to 
learning from errors. Debugging systems have this characteristic since 
their very purpose is to correct an unsuccessful system. But Fahlman'a 
"BUILD" program* also shares this characteristic. A plan to build a 
tower should not be discarded if the tower falls. Rather the beet 
option may be to fix it by using acaffolda or counter-weights. 

Sueeman's program is sufficiently sensitive to dead-ends that it 
compiles critiques to prevent repetition of unfruitful lines of 
development* 

Initially, Sussman's HACKER knows about planning and debugging but not 
about "gravity". It does not know that towers must be built from the 
bottom up. However, HACKER, In examining bugs due to towers falling, 

learns that building upwards Is essential. Interestingly, while the 



program learns this initially for towero of two blocks, It immediately 
generalizes to towers of any size. It Is comfortable with the concepte 
of "input" and "recursion"* 

Exploration, experimentation and improvement from mistakes Hill be the 
guidelines for a repairman rather than simple heuristic search. 



F. PLANNING 

Planning* whether to build a tower or to fix a program, often procedes 
in a top-down fashion. The general classification of the problem is 
made. The top level set of goals to solvs ths problem are generated. 
Then the system examines sach goal and decides how to satisfy it in 
turn. Difficulties occur whsn a sub-goal proves recalcitrant* The plan 
must be debugged. A system that has expertise in repair Is In a 
position to perform such a recursion, applying its skill to 1 ts own 
plan. Thus, the repairman is expectsd to sxhibit powerful planning 
capabilltes, which simpler non-self-critical programs intrinsically 
lack. 

i 
Fan I man has designsd an expert BUILDER for the BLOCKS-UORLO. This 
program ?a able to employ scaffolds and countsrwelghte to aid in its 
construction of towers, arenas and other structures. The important 
characteristic Is that It Is capable of first deriving a simple plan and 
later, when faced with "bugs", improving and augmsnting It* The program 
is leas an export on programs and sore a specialist in the physics of 
construction than Sussaan's HACKER program. However, both projects 
reveal how expertlet In planning and debugging greatly increases the 
power of a problem-solver, 

G. BOOTSTRAPPING 

The specialized expert* though the possessor of dstailmd and 
difficult knowledge, is oftsn incapable of changing domains* He lacke 
basic problem solving expertise In planning and debugging* expertise 
that has been structured abstractly so as to be generally applicable. 
The possibility that by abstracting the skills of debugging, 
bootstrapping will be possible In the design of a general repairman la 
fascinating. 



PROJECT 3.2 

AN AUTOHATIC PROGRAMING APPRENTICE 

For Softuare Production and Validation 

Carl Hewitt, Peter Bishop, Irene Grelf, 
Brian Smith, Todd Mateon, Richard Steiger 

A. INTRODUCTION 
i 
This proposal discusses the development of a programming system which 
understands what it Is doing. Us mean this in a surprisingly literal 
senses we propose to develop a system that understands not only the 
eteps of a program but what they are supposed to do and uhy. Such a 
system will help in construction of more Intelligent and powerful 
programs, will also serve in further understanding of reasoning, 
knowledge embedding, and Intelligence In programming. Although a highly 
Intelligent system cannot be built today. It Is a plausible long-tern 
goal because we can begin now with presently understood concepts- 
Recent research on Artificial Intelligence has given ue the tools and 
concepts for this. Including! 

Goal Oriented Formalisms 
Natural Language Semantics 

Source Language Interactive Debugging Systems 
Ability to confirm that a program satisfies Its Intentions 
Ability to make simple patches to procedures that fall to eatlefy 
their Intentions 

These Ideas need synthesizing Into an Integrated system. At our 
Laboratory the PLANtCR project [PLANNER Technical Report 3i "A Universal 
Modular ACTOR Formalism for Artificial Intelligence") has recently 
developed a coherent semantics which integrates many of these abilities. 

The apprentice we propose Is conceived as an initially "diligent but 
moderately stupid" apprentice to help in writing large programs. It 
will take the form of an Integrated edl tor-lnterpeter-debugger-problem 

eolver. Existing interactive debuggers (with the exception of 
Teltelman'e PILOT eystem, developed In our Laboratory) can only deal 
with syntactic aspects of programs. Thsy can catch misspelled words, 
correct the format of functional argumante, keep track of the use of 
functions, balance parentheses, etc. but can do little with eemantic 
content. Our initial goal is to be able to interactively answer such 
questions as "Uho called this function with a negative number?" or "Uho 
can come here with a list?". A subsequent stage will Include a eemantic 
model of the programming domain as well. 



B. AN INITIAL APPLICATION DOMAIN 

Ue are developing a language-system based on the idea of "ACTORS". 
ACTORS are universal modular computing unite that have a number of 
important advantages over conventional Mays of organizing programs and 
data. They communicate only through sending meseages In specific uaye. 
This is a eharp restriction -- compared to the free-for-all of 
Interactions permitted In ordinary programming systems. Our thesie la 
that Me gain much and loee little by using them. The implementation of 
actors on a conventional computer is a good initial domain in which to 
try out our ideas about automatic programming. Such a system will have 
to do the fol loulngt 

Trace implications of proposed changes in a configuration of 
actors. 

Determine the behavioral properties of other actors that a 
configuration of actors relies on. 

Trace the dependencies of some aspect of the behavior of a 
configuration of actore. 

Our apprentice must understand both programming and the domain dependent 
knowledge for which the program Is being written. First ue must teach 
our apprentice about programming In the area of ACTOR implementation. 

(The implementation of actors on a convsntional computer ie a large 
problem, not a toy one.) Ue have a number of experts on thie domain who 
are very interested in formalizing and extending their knowledge. Theee 
experts are good programmers and have the time, motivation, and ability 

to embed their knowledge and Intentions In the formalism. Once the 
experts put in some of their intentions, they find that they have to put 

in a great deal more knowlsdge to convince the auditor of the 
consistency of their intentions and procedures. In this way we hope to 
make explicit alt the behavioral ejuimpttau that our implementation ie 
relying upon. 

This will require neu representations for description of the system. 
Fortunately, thle domain is "closed" in the sense that the question* 
that can reasonably be asked do not lead to a vast body of other 
knowledge which would have to be formalized as well. It ie poeelble to 
start with a small superficial model of actors and build up 
incrementally. The task is simplified by excluding such complicated 
software engineering practlcee as the use of "go-tos*. Interrupts, or 
semaphores. 



C, KNOULEOGE BASED PROGRAMING 

"Knowledge based programming - It writing programs that have direct 
access to a substantial knowledge base in the application area for which 
the programs are intended. The actor formalism Mill aid knowledge- 
based programming In the following ways: 

PROCEDURAL EMBEDDING of KNOWIJZDGE has gained new impetus In recent 
years from the development of PLANf£R-like problem solving systems. 
Procedural knowledge enables us to put knowledge into a computer in 
a form such that it can be effectively used as intended. 

TRACING BEHAVIORAL DEPENDENCIES can be done by analyzing the 
intentions of programs and keeping track of each Intention that le 
relied on in another procedure* 

SUBSTANTIATING that ACTORS SATISFY their INTENTIONS can be done by 
binding procedures to their Intentions and meta-evaluating the 
programs. Meta-evaluation Is the process of reading the program 
abstractly to make sure that all of the Intentions of the actore 
that it calls are satisfied. 

"Testing examples shows the presence, not the absence, of bugs." 

In particular, our apprentice must be able to understand the following! 

The intentions that we express for our programs. 

The justification we give for believing that these intentione are 
satisfied by the programs. 

The behavior that our programs will exhibit if executed. 

Our apprentice digests this information as it is incrementally 
presented. It sometimes asks questions when it doesn't understand. 
Uhen we believe that we have finished writing the code and intentione 
for a configuration of actors we can ask our apprentice to execute it 
"abstractly" to see if It has a chance to work in general. Uhere It 
can't underetand the code it will try to give us high level feedback 
pointing to a place In the code and describing the nature of its 
difficulty. 

Ae a theoretical basis, we plan to try to uss Scott's lattice theory 
approach to recursion. In this theory, the msaning of a program can be 
understood to be the least fixsd point of a functional on a lattice of 
functions. Due to specific properties of these functlonale, thie fixed 
point can be constructed In a particularly nice manner. Induction over 
the stages in this construction is thsn the natural means of confirming 
facts about the fixed point and, thus, about the behavior of the 
program. 



Irene Greif is investigating how this interpretation of recursive 
programs can be extended to 'actor programs" or systems of actors. 
There is a natural minimal behavioral fixed point to any actor 
definition, but it le only understood informally. An extension of Scott 
logic to a theory of actors Mould not only formalize these concepte for 
actors but Mould shoM, Me hope, the value of taking a Scott logic 
vieupoint for more general systems. Actors can be used to define 
indeterminate systems as well as determinate ones and at present It le 
not known uhether these systems can be studied productively within the 
lattice theory framswork. 

One problem is to identify the lattice over uhlch ue Mill be defining 
actors. Since behavior nay be time-dependent* it may be neceseary to 
try to account not only for Input-output pairs but also for some 
relation over time. Thie wi 1 1 be a departure from the systems 
previously handled in this logic. 

This exteftelAn wi I i also yield rules of deduction for the neu logic. 
The rulee of deduction to establish that actors satisfy their intentione 
eeeentlally take the form of a high level interpreter for abstractly 
evaluating the program In the context of Its intentions. This process 
(called HETA-EVALUATION) can be Justified by a forn of Induction. To 
substantiate a property of the behavior of Bn actor system* some form of 
induction Mill be needed. At present, actor induction for an actor 
configuration ulth audience E can be tentatively described in the 
f o I I om i ng manner : 

If the actors In the audience E satisfy the intentione of the 
actors to Mhlch they send messages. 

And* if Mhen any actor's intsntions are satisfied, eo are those of 
all actors sent massages by it, 

Then the intentions of all actions caused by E arm satisfied (I.e. 
the eyetem behaves correctly). 

Greif Is investigating to see if this induction rule is related to the 
minimal behavioral fixed point in a natural uay. 

Thie uork should help us formulate precisely uhat a program does as 

opposed to hoM It does It. It gives us a natural* intuitive May to 

establish that a program does Mhat Is intended. And It provides a isore 
ootid foundation on uhich to build an apprentice system. 

A Simple Example 

Conerder the problem of writing a program to shift the geare of a truck 
with a manual transmission. Us apologize for the necessity for 



Introducing neu syntax but the following concepts ar« crucial 
discussion which follows! 



to the 



tx <-(-> ybody)] 

19 actor syntax which at a rough 
an actor r which, when it is cal 
is bound) executes 6orfy. 



intuitive level means: define 
ed with an argument (to which y 



(rules % (-> yl bodyl) (-> ?J 6*ly2)*.*l 
roughly neans: take *» and If It Batches yl t execute bodyli 
otherwise If It matches y2 % execute body!, etc.*. 

[* <*(intentlon (a) i/ definition 12)] 
is an elaboration of 1* meaning that when x Is called 
with a, then it is the intention of the incoming call 
and i2 is the intention when x calls out again. 



Our first try at a shift procedure night bet 

Primi tive-shl f t-toi when called with a target gear 
ia l t 2, 3, or 4 and calls the appropriate selectt 
left, upper-right, or lower-right respectively. 



checks to sea If It 
upper- left, I ower- 



Iprimltlva-shlft-to <- 

(■> target-gear (rules target-gear 
(-> 1 (select-upper-left)) (-> 2(eelect-Jower-lef t)) 
(-> 3<select-uppsr-right)) (-> 4(ee*ect-louer-r ight))) )J 

Now ue consider the various select routines and their intentions. Each 
of the select functions has an incoming intention that tha clutch be 
disengaged. Furthermore each of then has code (delimited by *) to do 
the selecting. Uhen a selector calls out, we fully intend for the truck 
to be in the gear appropriate to that selection. 



(select-upper-left <- 
(Intention M 
(clutch disengaged) 
*code-for-ae1 act-upper- left* 
(in-gear 1))] 

tse I act- lower- 1 eft <m 
( intention (] 
(clutch disengaged) 
ftcodo-for-select-upper-rtght* 
(in-gear 2))] 



[select-upper-right <- 
(intention [] 
(clutch disengaged) 
ivcode-for-select-upper- 
(In-gear 3>>3 



Ight* 



Iselect-louer-right <- 
(Intention I) 
(clutch disengaged) 
*code-for-ss I act- lower-right* 
(In-gear 4))1 



Our apprentice notices that for each one there Is a physical constraint 
that the clutch must be disengaged before shifting* He queries us about 
this and so ue decide to Modify the function PRIMITIVE-SHIFT-TO to first 
di eengage the clutch. 

[prlml tlve-shf ft-to <• 
(-> [target-gear] 

(disengage clutch) 
(rules target-gear 
(-> 1 (eelect-upper-left)) (-> 2 (eelect-louer-lef t)) 

(-> 3 (select-upper-right)) (-> 4 (eelect-lower-r ight) ) ) 

(engage clutch))] 
Now the code for prlel tl ve-ehi f t-to Is to first disengage the clutch, 
then do the selecting as before, and finally engage the clutch. 

Ue also write functione to dlssngags and engage the clutch* 

(disengage <- [engage <m 

(intention [clutch] (Intention [clutch] 

(clutch engaged) (clutch disengaged) 

frcode-for-disengage* *code-for-engage* 

(clutch dieengagad))] (clutch engaged))) 

Now our apprentice Is noil If led. However, the engineers dealing with 
the transmission cone to us with sons additional constraints* For 
example to select third gear the constraints Mrm now that the clutch 
muet be disengaged and the truck must be In either second or fourth 
gear. The other constraints are similar, 

leelect-upper-rlght <- (Intention 
(and (clutch dissngaged) (or(ln~gsar 2) (In-gear 4))) 
ttcode-for-eelect-upper-rlghtft On-gear 3))] 

(select-upper- 1 eft <- (Intention * 

(and (clutch disengaged) (stopped)) 
«ode-for-eelect-upper-lef t* (In-gear 1))] 

[select- lower -right <- (intention 
(and (clutch disengaged) (in-gear 3)) 
*code-for-se1ect-lower-right* (in-gear 4))] 

[oelect-lowsr-lef t <- (Intention 
(and (clutch disengaged) (or (In-gear 1) (in-gear 3))) 
*code-for-«elect-upper~rlghtft lin-gear 2))] 

The new requirements say that (teaporar I ly at Isast) the truck has to be 
etopped to shift into gear 1 and no gears can be skipped in ehifting 

while running. (Note: you can shift directly froti any gear to first If 
the truck is stopped.) So ue have to writs some new proceduree to neet 
theee new intentions. 



SHIFT-tos when called with a target gear considers, in order, the 
following rules for the target geari 

If It le first gear, then do a prtaltive-ehi f t-to flret 
gear. 

If It is either one greater than the current gear or one 
less than the current gear then do a pr le) t Ive-shl ft-to the 
target gear. 

If It le greater than the current gear then shift-to one 
less than the target gear and then pr 1 tn 1 1 ive-shl f t-to the 
target gear. 

If it I e lese than the current gear then ehlft-to one 
greater than the target gear and then pri»i tive-shi f t-to the 
target gear* 

[ehift-to <- (-> target-gear (rules target-gear 
<«> 1 (prialtlve-shlft-to 1)) 
(-> (either (current-gear + 1) (current-gear - 1)) 

(prl*itlve-ehlft-to target-gear) t 
(■> (greater (current-gear)) (shift-to (target-gear - 1>) 

(prieltlve-ehlft-to target-gear)) 
(-> (lese (current-gear)) (ehift-to (target-gear * 1)1 

(pr 1ml tivs-shlf t-to target-gear)))] 

Ue ask our apprentice to Mta-evaluate our prograa. It thinks for a 
while and sees two probleesi 

It can only shift to gear 1 if the truck Is stopped. 

It should not be aeked to shift to the gsar that it already 
te in. [The procedure shift-to doea not work If It is aeked 
to shift to the current gsar.) 

Ue decide to give the following intention to SHIFT-TOt If the target* 
gear ie flret gear then the truck must be stopped; otherwise the target- 
gear must be 2, 3, or 4 and not be the current gear. 

Cehi f t-to <■ (Intention target-gear (rules target-gear 
(-> 1 (stopped)) 

(*> (or 2 3 4) (target-gear /- current-gear)) 
(else (not-applicable))) 
*code-for-repeatedly-ehl f t-to* 
( in-gear target-gear))] 

To summarize ue have used intentions in the following soaewhat distinct 
wayei 



To specify what the actor la supposed to do ae opposed to 
how to do It* 

As a contract that the actor haa with Its external 
environment* How It carries the contract ie its own 
business* 

As a formal statement of the condition* under Hhich the 
actor Hill fullfill Its contract* 

The above example doss not dsal with all of ths computational Issues 
that our apprentice m 1 1 1 be faced with* For example It doss not have 
sophisticated data structurss and has no concurrancy or parallelism. 
Consider an on-line data base for an air traffic controller* Ue shall 
suppose that ths data-base contains ths position and amount of fuel left 
for each plane that is currently airborn. Various processes will want 
to read from and write Into this data base* It is important that a 
process not get inconsistent Information when it reads out the poeition 
and fuel of a plans. This mlflht happen if another process ie 
concurrently updating the position and fuel of a plane* Inconsistent 
Information might result in a plans crash because the controller makes 
its decieions on the baeis of ths information it rsads in the data base. 
So ue need to put a scheduler in front of the data base to allow only 
one process to write In it at once. The actor formalism enables us to 
deal with problems like this In a straightforward way. 

Peter Bishop is investigating the feasibility of an actor machine. 
There are a number of unsolved problsns both at ths level of 
architectural design and efficiency. 

For example, protection is an Important issue that must be faced by next 
generation systems. Actors provide a degrss of intrinsic protection. 
There Is no way to coerce an actor Into doing anything that it doesn't 
want to do. Thus* at a certain level actore can be passed around quite 

freely since they will only work for authorized users. Futhermore an 
actor only knows about the actors that It has been sent as messages. By 

these means it appears that actors can implement all of the proposale 

for protection mechanisms that havs thus far been published* More work 

Is necessary bsfors we will know how intrinsic protection is best 
utilized. There remain soms important problems in protection Involving 

intent and trust. Ue are currently considering ways In which hardware 
can be further developed to address the problems. Another important 

Issue Is retention of storage. Current garbage collection techniquee 
are not very efficient if ths amount of storage retained is very much 

larger that the amount of faet random access memory on the machine. 
Knowledge based systsms will require a vast amount of on-line storage. 
Ue uill be investigating techniques for making garbage collecton 

feasible or unnecessary under such circumstances. 



D. PLANNER PROGRESS 

This section gives a feu more details about feature* of the 
proposed neu ACTOR-baaed PLANNER system. 

The PLANNER project is continuing research in natural and effective 
meane for embedding knowledge in procedures. In the course of this Mork 
mo have succeeded in unifying the formalism around on* fundamental 
concept! the ACTOR. Intuitively, an ACTOR is an active agent which 
plays a role on cue according to a script. Data structures, functions, 
semaphores, Monitors, ports, descriptions. Qui Mian nets, logical 
formulae, numbers, Idsntlflers, demons, processes, contexts, and data- 
bases can all be shown to bs special cases of actors. Our formalism 
shows how all of these modes of behavior can be defined in terms of **# 
kind of behavior: tending mettaget ts acton. An actor le always ■ 
invoksd uniformly In exactly the same way regardlsss of whethar It 
behaves ae a recureive function, data structure, or process- 

The unification and simplification of ths formalisms for the procedural 
embedding of knowledge has many benefits: 

INTENTIONS! The confirmation of properties of procedures Is made easier 
and more uniform. Every actor has an INTENTION which checks that ths 
prerequisites and ths contsxt of the actor being sent the message are 
satisfied. By a SIMPLE BUG ws mean an actor which doss not satisfy Its 
intention. Ue would like to eliminate simple debugging of actors by tho 
ftTA-EVALUATION of actors to show that they satisfy their intentions. 
To do this ue have a proof-method called ACTOR-INDUCTION. Computational 
induction fllannal , etructural Induction (Gurstalll, and Peano induction 
arm all special cases of ACTOR Induction. Actor bassd Intentions have 
the following advantages! 

The intention Is d*c*mpM from the actors It describes. 

Intentions of eeneiuT*** actions are nore easily disentangled. 

Ue can more elegantly write intentions for dialogues between actors* 

Tho intentions are wrlttsn In the Mine formalism as the procedures 
they describe. Thus for example Intentions can have intentions. 

Because protection is an intrinsic property of actors, wa hope to be 
abla to deal with protection issues in the same straightforward 
manner as more conventional Intentions* 

Intentions of data structures are handled by the mbw machinery ae 
for all other actors. 



COMPARATIVE SCHBM ATQLQGV I The theory of comparative power of control 
structures is extended and unified* Ths folloMlng hierarchy of control 
structures can be explicated by incrementally Increasing the power of 
the actor message sending primitive: 

i terattve — >recurslve — >back track — >de tormina to — >unlvsrsal 



EDUCATlONi Ths model Is sufficiently natural and elmple that i t can be 
made the conceptual basis of ths model of computation for students- In 
particular It can be used as ths conceptual model for a generalization 
of Seymour Papert's "little man" model of LOGO. The model becomes a 
cooperating society of "little men" each of whom can address others Mlth 
whom it is acquaintsd and politely request that some task be performed* 

EXTENDABILITYt The model provides for only one extension mechanlsmt 
creating new actors. However thle mechanlem le sufficient to obtain any 
eemantlc extension that might be desired. 

PRIVACY ANO PROTECTIONx Actore enable us to define effective and 
efficient protection schemes* Ordinary protection falls out as an 
efficient intrineic property of the model. The protection is based on 
the concept of "use". Actore can be freely passsd out since they nl 1 1 
uork only for actora which have the authority to uss them. Mutually 
suspicious "memoryless" subsystems are easily and efficiently 
Implemented. ACTORS are at least as powerful a protection mechanlem as 
domains tSchroeder. Needham, etc.), access control llete [fflJLTICS], 
objects lUulf 1972) and capabilities [Dennis, Plummer, Lamp son] * 
Because actors are locally computationally universal and cannot be 
coerced, there is reason to believe that they are a vmtwcoi protection 
ntecAanitm in the sense that all other protection mechanisms can be 
efficiently defined using actors. The most important iesues In privacy 
and protection that remain unsolved are those involving intent and 
truet. Ue are currently considering Mays In uhlch our model can be 
futher developed to address thess problems. 

SYNCH RONIZATIONx It provides at least as powerful a synchronization 
mechanism as the ntultipls semaphors P operation with no busy waiting and 
guaranteed first in first out discipline on each resource. A 
synchronization actor is easier to use and substantiate than a multiple 
eemaphor CDijkatra 1971] since they are directly tied to the control- 
data flow. 

SIMULTANEOUS COALSi The synchronization problem is actually a special 
case of the simultaneous goal problam. Each resource which Is seized Is 
the achievement and maintenance of one of a number of simul tansous 
goals. Recently Sussman has sxtended the previous theory of goel 
protection by making the protection guardians into a list of predicates 
which must be evaluated every time anything changee. Ue have 



generalized protection in our model by endowing each actor with a 
scheduler and an intention, Ue thue retain the advantages of local 
intentional semantice, A scheduler actor allows us to program EXCUSES 
for violation In case of need and to allow NEGOTIATION and re- 
negotiation between the actor which seeks to seize another and its 
scheduler, Richard Ualdinger has pointed out that the task of sorting 
three numbers ie a very elegant simple exanple illustrating the utility 
of Incorporating these kinds of excuses for violating protection, 

RESOURCE AtLQCATlONi Each actor has a banker who can keep track of the 
resources used by the actors that are financed by the banker. 

STRUCWRINCt The actor point of view raises some interesting questions 
concerning the structure of programming, 

STRUCTURED PROGRAMS* Ue maintain that actor communication Ie well- 
otructured. Having no "go-to," Interrupt* semaphore, or other 
constructs, they do not violate "the letter of the law". Some 
readers will probably feel that some actors exhibit "undisciplined" 
control flow. These distinctions can be formalized through the 
mathematical discipline of comparative schematology [Paterson and 
Hewitt). 

STRUCTURED PROGRAMMING* Some authors have advocated top down 
programming, Ue find that our own programming style can be mora 
accurately described as "middle out". Ue typically start with 
specifications for a largs task which ua would like to program. Ua 
refine these specifications, attempting to create a program] as 
rapidly as possible. This initial attempt to meet the 
specifications has the effect of causing us to change the 
spec! f icatons in two ways: 

li flora specifications [features which ue originally did not 
realize were important) are added to the definition of the task. 

2i The specifications are generalized and combined to produce a 
task that is easier to implement and more euited to our real 
needs. 

IMPLEMENTATION! Actors provide a vsry flexible implementation language. 
In fact we are carrying out the implementation *itttr#ly In the formalism 
itself. By so doing ue obtain an implementation that is efficient and 
hae an effective model of itsslf. The efficiency Is gained by not 
having to incur the interpretive overhead of embedding the 
implementation In some other formalism. The model enables the formalism 
to answer questions about itself and to draw conclusions as to the 
impact of proposed changes in the Implementation. 

ARCHITECTURE* Actors can be made the basis of the architecture of a 
computer which means that all the benefits listed above can be enforced 



and made efficient. Programs written for the machine are guaranteed to 
be eyntactically properly nested. The batlc unit of execution on an 
actor machine le sending a message much In the same way that the basic 
unit of execution on present day machines ts an instruction. On a 
current generation machine* In order to do an addition, an "add" 

instruction must be executed; so on an actor machine a hardware actor 
must be sent the operands to be added. There are no goto, semaphore* 

Interrupt* etc. Inetructions on an ACTOR machine. An ACTOR machine can 
be built using the current hardware tschnology that is competitive with 
current generation machines. 

Hierarchies 

The model provides for the following orthogonal hlerarchiee: 

SCHEDULING* Every actor has a scheduler which determines when tha 
actor actually acts after it is sent a nessage. The scheduler 
handles problems of aynchronliat Ion. Another job of the echeduler 

IRulifeon) is to try to cause actors to act in an order such that 

their intentione will be satisfied. 

INTENTIONSt Every actor has an intention which makes certain that 
the prerequisl tee end context of the actor being eent the meeeage 
are satisfied. Intentions provide a certain amount of redundancy 
in the specification of what ie supposed to happen. 

MONITORING* Every actor can have monitors which look over each 

messags sent to the actor. 

- 

BINDINCt Every actor can have a procedure for looking up the 
values of names that occur within it* 

RESOURCE MANAGEMENT* Every actor can have a banker which monltore 
the use of space and time. 

Each of the actlvltlee is locally defined and executed at the point of 
Invocation. This allows the maximum possible degree of parallelism. 
Our node I contrasts strongly with extrinsic quant i fleet ional calculus 
models which are forced Into global noneffective statemente In order to 
characterize the semantics. 



PROJECT 4 
NATURAL LANGUAGE SYSTEMS 

DEFINITION; Our goal in this area is to team how to make scotoma that 
understand language in deeper and more effective ways* 

More specifically, we plan to develop interactive natural language 
system* for use with the performance systems described elssuhere In this 
proposal. An Internationally known program developed in this laboratory 
by T, Uinograd represented a direct and Intense effort to combine both 
new and old theories into a single performance system. A next step is 

to "push ahead" to make this system mors powerful, and several Morkers 

are doing just that* 

However, many very serious problems remain. Uinograd and others working 
in this area solvsd some of the most persistent problems of 
computational linguistics by using 'procedural methods", in uhich one 
can Mr i te special computer programs to handle all sorts of 
contingencies. But this Isads to a dilemma. Although It eeeme easier 
to handle any particular difficulty, the whole system grows hard to 
understand because of neu and dl ff Icul t-to-describe kinds of 
Interactions. Furthermore, parts of ths system become Inaccessible to 
"self-conscious** operation — they are hidden in the system code, away 
from the deductive mechanisms. The system becomes unable to explain Its 
actions, either in dlrsct answer to questions or — worse — even In 
response to debugging probes by systems programmers. 

For this reason, we cannot depsnd entirely on the "holistic" approach ™ 
that Is, of making one eingls complete demonstration and experimental 
system -~ ue must also consolidate uniformities in representations* The 
traditional approach of grammarians, to put things into uniform, 
declarative structures, makes neater those concepts that can be so 
expressed, but their inflexibility has always led to Inadeqate power. 
Ue plan to push in a variety of directions to gst better control of a 
great many ideas about procedures, and ssmantlcs, annotation and 
compilation, and ssveral specific questions about ths relatione of 
llnguietic etructures to heuristic modele of the world. 

MILESTONES! 

This is a complex, rapidly developing field. There is close interaction 
between many different laboratories, and systems are being ehared ovtr 
ths ARPANET* 

Ue expect to see some of ths sffscts of this research on the Natural 
Language milestones mentioned in Projscts 1, 2, and 3 above, within 
three years* There are different syntactic requirements and ssmsntic 
representational and rsasonlng problems In each eyetem. It is 
appropriate for natural language research to alternate between theory 
and experiment* This Is the way we have proceeded in the past. In the 
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works of Bobrow, Raphael, and Uinograd, and we expect that style to 
continue to be productive. Horn are some of the steps we expect to 

in the next three years. Ue cannot be more preclee about dates, but 
most of these &re involved with Ph. D. theses already In progress, and 

three years has been our Mean time from start to finleh of such 
projects. 

U. Martin — Natural language system based on new ideas about case- 
frames. This is a major enterprise in Project MAC. A nuaber of 
workers In our laboratory »rn working closely with Martin and hie 
group in this project* 

V* Pratt — a metalanguage for describing classical parsers. Uith 
facilities for incremental changes, and uas of local variables In 
the parser. The prototype system will describe rules of 
conjunction and elision. 

F1, Marcus — Concept of "wai t-and-see" parser. Some parts of language 
sre more rigid than others. The order of things within English 
Noun Groups are relatively inflexible* as compared with the 
ordering of constituents Mlthin a clause. The wait-and-see pareer 
should be able to exploit these Inhomogenel ties to get more 
efficient, yet still orderly, parsing programs* 

D. McOermott — attempt to uniformize knowledge about "doubting" and 
plausibility. Attempt to uniformize "expert" knowledge for 
compilation into procedures, uith declarative representation 
avai I able for deduction* 

A. Rubin — ReJationlbetyeen conventions of tins and tense in English 
to assumptions about qualifiers and quantifiers. Use of "usually", 
"probably". The sentence "I teach on Uednesday" is really a future, 
"I teach on Wednesdays" is a (chronic) future" even though on the 
surface these use present tense. The iatter means "I usually -*-*■ 



R. Moore — In a somewhat similar vein, R. Moore is studying, from a 
procedural point of view, questions about referent lal I ty and 
"knowledge about knowledge". Some of these problems have been long- 
term bugaboos in classical declarative logic, and have never been 
carefully treated in connection with computational linguistics. 

0, McDonald — An "advice-taker" eystem for interaction In Engl I eh 
between a chess program and a chess expert. 

PROBLEMS: 

How to build a metalanguage for describing modern "grammars". 
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The neu heterarchical linguistic models do not submit to prestructured 
■syntax-directed' methods. 

Uhat should be the etrategy with respect to ambiguities in parsing now 
that semantic methods are beginning to exist. 

Can we find reasonably regular ways to describe the neu heuristic 
parsing schemes that use larger structures: scenarios, frames, 
etc? 

Moat important to us zr% questions related to the issue of "uniformity". 
There is a constant struggle, in both linguistic and deductive research, 
between schemes that appear to neatly formalize an area of knowledge in 
a set of relatively orderly principles and rules — vs. "exceptions" 
that become critically important in real applications and clash uith the 
broad uniformities. Can we make a system that can exploit the 
advantages of the regular systems (easy to debug, easy to augment by 
other loosely associated workers, etc. I within a framework that can 
handle exceptions also in an "orderly", higher level way? 

PERSONNEL: V. Pratt. R. Hoore. A. Rubin, 0. McDonald, H. Marcus: 
others who will become involved (this is a popular and growing area), 
and collaboration with U. Martin's group In Project MAC. 

COSTS: flajor Computational Resources Required. Systems use languages 
LINGOL, niCRO-PLANNER. PftOGRADtlAR, LISP, CONNIVER, PLANNER. HIOAS and 
possibly others. Natural language programs saturate our current 
computer system when running, and growth of thie area will require 
expansion of primary computer memory. 
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PROJECT 4 
NATURAL LANGUAGE RESEARCH 



A. INTRQCLCTIGN 



Let ua pretend that It It possible to decompose English conversation 
into processed of Listening, Thinking and Speaking. In large measure* 
it Mas the erasure of such distinctions that was responsible for the 
great progress of the past feu years. But traditional frameworks are 
etill useful in presenting an overall view of what is happening. 

Bobrow's program, STUDENT, was art early product of this laboratory that 
exhibited all of these components cooperating on high school algebra 
problems. Uinograd's BLOCKS program, SHRXU* If a considerably more 
sophisticated approach to the same issues, with a much deeper connection 
between details of the structure of natural English and the meanings of 
words* clauses* and whole discourses. Our experience with SHROLU has 
had two important consequences: 

It has shown us that quite non-trivial natural language programs 
can be written with today's hardware and software. This 
demonstration has encouraged a fresh burst of natural language 
research, particularly in this laboratory. 

It has pin-pointed the problems we oust deal with in producing a 
successor to SHROLU. 

It le appropriate for natural language research to alternate between 
theory and experiment* SHROLU represented a direct and intense effort 
to combine the accumulated theory into an experimental progran. Ue 
intend now to focus on the theoretical implications of SHRDLU's good and 
bad features, with the hope that what we learn will be applicable in the 
near future to another large program that will cope with many Issuee not 
addressed by SHROLU. 

Ue propose to divide our attention between listening* thinking and 
speaking* deferring for the time being the issue of getting the results 
of our efforts to interact gracefully. The reason for this temporary 
de-emphasis of the "holistic" approach to natural language is the need 
for considerable attention to theoretical detail before ue will be ready 
for the next SHROLU* 

B. LISTENING 

Under the rubric of "listening" we include everything necessary to 
convert typed input into a representation convenient for the "thinking" 
experts to work with. The immediate difficulty here is that it is 
unclear what that representation ought to be. 
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Vaughan Pratt and ttitcheil Marcus are studying some aspects of parsing. 
Pratt is concerned with the representation of grammatical knowledge - 
what ie an appropriate language for describing English to people and/or 
parsing programs? Marcus is designing a parser whose overall strategy 
takes advantage of certain properties of English, 

Representation of Grammatical Knowledge - Vaughan Pratt 

A programmer is free to impose as much or as little structure on hie 
programs as he pleases. The advantage of having little structure ie 
that he can attend to the problem of encoding his algorithm with a 
minimum of constraints on how he may express himself. Unfortunately 
this style of programming frequently leads to obscure programs, with 
some undesirable consequences! 

The program Is hard to debug. 

It is hard to tell what information has been represented in the 
program. 

The first consequence may be the programmer's own private problem. When 
the program proves successful, however, as SHRDLU did, computational 
linguists, in order to to build on the original program, may have to 
start from scratch. This is a major difficulty with ShfiOLU at present. 
The second consequence is a problem that arises when the programmer 
wants to claim that his program in some sense describes the properties 
of its problem domain. To date this has not been a serious problem for 
natural language programmers because no program yet exists that contains 
enough information to constitute a better manual of English than those 
that already exist on bookshelves. This situation may change with the 
descendants of SHRDLU, and so is worth anticipating. 

Historically, the first parsers were relatively unstructured. Later, 
context-free grammars became popular, and imposed their own peculiar 
brand of structure on parsers. Kuno's Predictive Analyzer represents 
the zenith of that era. In 1967, Thorne experimented with a more 
procedural approach to representing grammatical knowledge, and was 
quickly followed by Bobrow and Frazer. Uoods, and Uinograd. In the 
process, the flexibility conferred by the procedural approach allowed 
programmers to write code for each problem as it arose, to the detriment 
of any structure they or their programming language intended to impose. 
We seem to have witnessed the rise and fall of structured programming in 
parsers. 

Pratt is exploring one approach to this apparent malaise which may 
combine the advantages of structured and unstructured approaches to 
parser writing. One wants to retain the advantage of being able to 
solve any problem juet by generating the appropriate code, without 
losing the clarity conferred by a more structured style. The approach 
suggested is to start out with a metalanguage for describing parsers 
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which It designed to deal elegantly with the known issues In parsing 
natural language, and then not freeze the design of this metalanguage 
but rather let it groM in response to unanticipated needs. For this to 
be successful, growth will have to be slowj if it turns out that the 
process of adding descriptive mechanisms to the metalanguage does not 
converge rapidly, then we ar% no better off than with a completely 
unstructured approach. 

There are at least four possible pay-offs from such an approach: 

It will be easier to write parsers from scratch, and to understand 
and extend other peoples* parsers. 

It will be clearer what our parsers have to say about English; that 
is, the parser itself »ay constitute a viable grammar of 
Engl i sh. 

The metalanguage that evolves may turn out to be a valuable 
descriptive tool for linguists independently of its 
application to computers. 

The structure of the evolved metalanguage will In Itself provide 
insights into the nature of English, in that it will be a 
source of generalities about different kinds of linguistic 
phenomena. 

On* fault with previous attempts to provide a conplete descriptive 
language, such as Chomsky's transformational grammar, is that they did 
not explicitly provide for growth, and so stagnated. The situation ie a 
little like that of a carpenter who initially sees a need for a hammer* 
saw and screwdriver, and from then on tries to do everything using only 
those tools. By allowing room for growth, we nay avoid the situation in 
linguistics where one school of thought correct* for inadequacies in 
another's metalanguage by abandoning it completely and replacing it uith 
a radically different on©, which makes It difficult for the differing 
schools to build on each other's work. 

A similar situation is obtained in programing language design 
philosophy. One school of thought offers PL/I as a complete panacea* 
while another prefers the notion of an extensible language on the 
grounds that we ar^ unlikely to anticipate every programming need* It 
is possible that some of the ideas that have arisen in the development 
of extensible languages may be transferable to the problem in h* 1 

A serious difficulty with "growing" a metalanguage is that nothing a' 
all is known about how to do this gracefully. Pratt has been using 
LINGOL. a programming language designed for linguists, as a medium in 
which to carry out some experiments with this approach. The results so 
far have indicated that smooth growth can be achieved only by playing it 
by ear. At present the process seems to consist of discovering 
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Inductively the most general form of what one wants to say, and then 
designing the appropriate metalanguage feature. An example of such a 

feature is the use of the notion of local variable In the surface 
structure of English sentences. This feature was readily implemented by 
drawing on the corresponding feature in LISP. It seems to be a 
generally useful idea, being applicable to a large variety of problems 

in negation, as well as to difficulties in subject-verb and adjective- 
noun agreement when translating into, say, French or German* 

A project that may test whether this Idea of growth can be carried 
further ie that of finding ^n appropriate way to describe the rules of 
conjunction and elision, as in "The Chinese have short names and the 
Japanese long". Further experience with such problems hopefully will 
lead to a better understanding of the value of this approach, and may 
load to more systenatic methods of growing metalanguages. 

Hal t-and-See Parsing - flitcheM Marcus 

Ulhen a parser is faced with a choice of possibilities. It nay choose 
one, proceed, and if difficulties are later encountered, back up to the 
point where the decision was made and choose another possibility. 
Alternatively, It may choose all possibilities simultaneously, and carry 
along all of their consequences In parallel, hoping that sooner or later 
most of the possibilities will fall by the wayside. In the early days 
of parsing, only the first option occurred to programmers. Then Cocke 
suggested an algorithm which was adopted by Kay, which implemented the 
second option. At the time it was felt that the parallel method was 
less efficient than the back-up method, which Kuno proceeded to use in 
the Harvard Predictive Analyzer, In 1387 Younger and Earley 
independently pointed out that in principle the parallel approach was 
really considerably more efficient than "destructive" backup, in which 
the result of parsing a substring is abandoned when backing up over that 
substring. Kuno changed his parser to incorporate non-destructive 
backup and reported order-of-magni tude improvements. Thome's (1967) 
procedurally oriented parser used the parallel-parse approach also, but 
from then on Thome's successors reverted to the backup aethod in the 
hope that by guessing cleverly, backup could be held to a minimum. So 
far no one has claimad that his parser guesses right most of the tl«e. 

Harcus proposes to combine the general Idea of parallel parsing with 
techniques that take advantage of certain features of English. It 
appears to be the case that some rules of grammar are more robust than 
others. In particular, the rules that dictate word order within Noun 
Groups are quite inflexible. For example* one may say fc my five handsome 
hunting dogs H , but not "my handsome hunting five dogs" or "»y five 
hunting handsome dogs". Moreover, It Is not even a question of style; 
the meaning of the phrase can be distorted or lost by such a 
permutation. The situation with Verb Groups such as "should have been 
seen" is even more rigid with respect to word order, although adverbs 
&re permitted to appear within the group, preferably after the first 
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word of the group. 

In contrast, the order of clause constituents (Noun Groups, Adverbs, 
Verbs) is relatively unimportant. Adverbs may appear anywhere, although 
they may not break up already formed Noun Groups. Subject and Object 
tend to go before and after the verb respectively, but this rule is 
frequently broken, e.g. "Some flavors almost everybody likes", and 
poetic license readily allous "Heard I the lark at eventide" while 
tending to frown on interference with Noun Group rearrangement, other 
than to allow adjectives to follow nouns (a variant also permitted 
occasionally in prose). 

These observations suggest that a fairly conventional syntax-based 
approach may be adopted for parsing Noun Groups, and that a more 
semantical iy oriented technique is appropriate for elucidating the 
relations holding between the clause constituents* 

This motivates the notion of the Uait-and-See parser. Uhen a Noun Group 
is being scanned, the parser commits itself immediately to the internal 
structure of this Noun Group, and also attempts to determine its 
semantic referent in order to facilitate assembling clause constituents* 
Ambiguities within a Noun Group are recorded as part of the structural 
Information about the Noun Group and are thus an internal problem and 
only marginally relevant to the clause constituent assembler. 
Ambiguities about where the Noun Group begins and ends are recorded as 
separate Noun Groups, to be sorted out by the clause constituent 
assembler. Verb Groups are handled similarly; ambiguity tends to be 
less of a problem with Verb Groups than Noun Groups, and so the 
situation is even simpler. Other types of clause constituents are for 
the most part relatively slftple and require only cursory inspection In 
order to Identify then as clause constituents. 

The part of the parser describsd so f^r pay be characterized as a 
"*iddf e-doun" parser, as distinct froe the usual top-down and botto«-up 
parsers. It is niddle-down because initially it is on the look-out for 
clause constituents, and can generally manage to predict the type of 
constituent from looking at its first word* The commonest exception to 
this is in the case of Noun Groups that don't begin with a determiner 
and follow another Noun Group, as in "The city people like is Boston". 
In this and similar cases, the parser preserves its middle-down flavor 
by guessing all possibilities as early as possible* 

As clause constituents sre discovered by the first component of the 
parser, a second component, the one that is going to draw on semantic 

information, attempts to fit these constituents together to form 
clauses. In the absence of rigid syntactic rules at this level, this 
component will frequently have to allow several constituents to 
accumulate while it waits to see (hence the tere Uait-and-See parser) 
what ie the most plausible way of putting the constituents together* It 

Is at this level that considerable emphasis will be placed on clever 
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methods to use alt available syntactic, semantic and contextual 
information to complete the parsing of the sentence, flany problems 
related to conjunction and elision ulll also be handled at this level, 
and Marcus hopes that his approach will considerably simplify these 
problems. 

The wait-and-see approach can be seen to differ in important respects 
from both backup-oriented and parallel parsing. Certainly no backup is 
involved. On the other hand, leaving clause constituents lying around 
unattached until their role can be determined differs from the parallel 
approach In that the latter attempts to commit constituents to all of 
their (possibly multiple) roles at the sams tine. The wait-and-see 
approach commits nobody until a definite role emerges. 

flarcue proposes to implement a parser based on these ideas starting at 
the end of the current summer. 



C. THINKING 

"Thinking" embraces a wide range of activities and problems. A listener 
may try to figure out what the speaker really meant by his utterancei hm 
may search for inconsistencies in iti he nay deduce its consequences) 
he way attempt to reply to it: he may take physical action based on iti 
he may draw conclusions about the speaker; or he nay simply choose not 
to believe a word the speaker is saying. Obviously this is not intended 
to be an exhaustive list - it just illustrates the complexity of the 
"thinking" problem domain* 

Consistent with this complexity, we have most of our people working on 
problems in this area. Drew llcOermott is following up the ideas 
developed in hie Master's thesis* and is concerned with the problems of 
representing and updating knowledge; the degree to which one can 
successfully carry out the tatter depends heavily on the nature of the 
former, and eo work is needed on suitable rspresentat ions. Andee Rubin 
Is addressing the interplay between time and tense, nominally English 
has provision for dealing with past, present and future events, but In 
practice relying on tense alone may give quite erroneous information 
about time. Robert Moore is developing a language suitable for 
representing a number of concepts that arm dealt with poorly or not at 
•II by other formalisms such as the predicate calculus. 

Representing and updating knowledge - Drew McDermott 

The notion of procedural embedding of knowledge has been popular for 
some time in this Laboratory. Drew McDermott has Just coraplsted a 
Haeter's thesis in which hs explored this concspt In considerable depth, 
by representing as procedures all the knowledge in TOPLE (for TOPLEvel), 
a natural language understanding system. McDermott is dissatisfied with 
some aspects of procedural embedding. 
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One problem that arose in TOPLE Mas that different types of knowledge 
varied in their accessibility by TOPLE. Some knowledge was etored as 
CONNIVER items* These are pattern-accessible pieces of knowledge, 
Thinge like (SLIGHTLY-BIGGER BOX! B0X2) Mere stored this way. However, 
knowledge traditionally requiring quantifiers, like, "if x is slightly 
bigger than y, and y is slightly bigger than z, then x is bigger than 
z, " were hidden aMay in procedures and ad hoc data tables. Such 
desertions were not in the eame format as the items, and could not be 
meditated upon at all by TOPLE- Still other forms of knowledge, such 
as, "if s is a step in the proof of p, and you want to doubt p, try 
doubting a, 11 were even more remote, being distributed throughout the 
eye tern. 

If we assume that a program that "knows itself" is going to be able to 
make more effective use of what it knows, it would seem reasonable to 
elevate all knowledge to the same level. This seen* to lead back to the 
Idea that knowledge should perhaps after all be represented as 
declaratives. Unfortunately, one always needs some procedure lurking 
somewhere In order to make things work, and so we appear to be in a bit 
of a dilemma. tlcDermott plans to sort out these issues in the not too 
di stent future. 

For the present, McDermott is considering altsrnatlve repreesntatlone 
for belief systems similar to TOPLE* The types of knowledge that need 
to be represented include* 

Knowledge about particular domains. This is the knowledge about space, 
what monkeys can do, etc. In TOPLE, It is scattsred about various 
CONNIVER methods. 

How to uss such Information, For sxample, the different partial 
orderings (such as sizs latticss for physical objects, space 
relationehlps) ar^ ssarched by expert routines rathsr than by a 
general deductive system, A different example Is the monkey 
eimulator by which TOPLE understands the monkey in its world. The 
knowledge about what a monkey can do Is scattered about and 
interleaved with knowledge about what he is likely to dOt 

How to doubt things. Each routine that adds a piece of knowledge to the 
world model is responsible for saving with it the information 
needed to remove It later should It conflict. There ie no reason 
why thle Information shouldn't be expressible. 

Uhat changes ars bsttsr than others. This is done with entirely ad hoc 
nunbere and whistles in the current version, and should definitely 
be systemat ized. 

How to debug statements with quantifiers. In the current TOPLE, all 
euch statements ere buried in routines which the system ie not 
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•mart enough to examine. Thar* is no reason why a formal language 
cannot ba discovered which can sxpress hou to debug statements In 
tha Janguaga* 

Tha mora knowledge ia ancodad in declaratives, the more the belief 
system will have to ba reading instructions and following tham or 
figuring tham out, rathar than just doing them, and the slower things 
yill go. This ia especially true if, say, knowledge about how to go 
about deducing la stated In a deductive formalism. A most promising 
approach seems to be to express everything in declaratives and compile 
tha world model periodically using the nethods studied by Sussman here 
at tl.I.T. He has a BLGCKS-JORLD systea with knowledge about 
programming, debugging, and physics, which compiles atatements about 
blocke and actions into programs to carry tham out* McOermott Intends 
to atudy hie work very carefully to sea how It night be converted to 
thinking about doubting beliefs instead of moving blocks, and to eea to 
what degree the kinds of knowledge his progran has can be expressed in a 
unlfora language. Than perhaps his program can be used to compile 
general daclaratlvaa Into expert procedures. 

Time and Tense - Andee Rubin 

Thla year Rubin proposes to continue hsr Investigation of natural 
language, concontratlng on tha ssmantics of tiros expressions* A 

preliminary goal will be to understand the syntax of the English tense 

system; this structure has already been investigated, as evidenced by 
the work of Halliday and Bruce. However, knowing the tense of a clause 
is often not enough; in many cases it is completely misleading. Even 
the eimplaet of taneee, present tense, has a myriad of usee, spanning 

past, present and future. Consider these examples! 

Care use gae. (universal) 

Alphonse loves his doctor. (present) 

Alphonse sees his doctor Uednesday. (future) 
Alphonse sees his doctor Uedneedays. (future) 
If he goes, I'll go. (future conditional) 

Even sentences which appear at first glance to contain preeent tense 
both syntactically and semantical ly, may have this meaning changed by 
context. A secretary entering an office with official notices who asks 
"Hou many people are in this office? 1 * doesn't expect a reply uhich 
elmply requires looking around and counting. Her question might be 
rephraeed ae "How many psople are usually In thla office?" Time 
conaiderat ions are not immune from the whole "usually" problem how to 
do logic with qualifiers such as usually, probably and typically. 

On the representational side, time will not be organized as a uniform 
set of points linearly ordered by the relation "before." Rather, time 
will be in chunks, such as YESTERDAY* TH1S-YEAR. THlS-flORNING etc.. 
which will be in general hierarchically organized. Evente will be 
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partial lg ordered within context** but certainly no total ordering can 
be expected* In addition, certain subcontexts may have a predetermined 
internal structure; we tend to organize workdays Into HORNING, LUNCH, 
AFTERNOON, DINNER and EVENING and the time of events is thus often 
specified as "after dinner" or "before lunch." 

This research uill be carried out in the mini-world of everyday 
activities, euch as going to school, shopping, going to work, traveling 
etc. The program will be a virtual roommate which accepts information 
about the past, present and future activities of its roommates and 
answers questions about their whereabouts, plans and schedules. It is 
hoped that this context uill provide plenty of opportunities for using 
and deciphering more complex time and tense references. 

A Computational Theory of Descriptions - Robert Moore 

Methods advocated for representing knowledge In Atlflclll Intelligence 
programs have included logical statements (McCarthy, Sandeuall), 
semantic networks (Quit I Ian, Schank) , and procedures (Hewitt, Suesman 
and McQsrmott). All these approaches share one fundamental concept, the 
notion of predication. That is, the basic data structure in each system 
is some representation of a predicate applied to objects. In this 
respect, the various systems are more or less equivalent. But this 
basic idea must be extended to handle problems of quantification and 
knowledge about knowledge. H^ro the systems do differ although these 
differences result from the descriptive apparatus used in the particular 
systems being compared, rather than from an inherent advantage of, eay, 
procedures over declaratives or vice versa. 

Advocates of PLANNER (e.g. Uinograd* p. 215) have argued that the 
predicate calculus cannot represent Q&u £ piece of knowledge should be 
used. But this is true only of the f ir?t-order predicate calculus. In 
a higher-order or non-ordered declarative language, statements could be 
made which would tell a theorem prover how other statements are to be 
used. PLANNER, on the other hand, has no way of directly stating an 
existential quantification, but this does not msan that procedural 
languages are necessarily incapable of handling that problem. Moore hae 
worked out the prel Iminary detai I s of a language D-SCRIPT with powerful 
formalisms for descriptions, which enables it to represent statements 
that are problematical in other systems. Since it is intended to answer 
questions by making deductions from a data base, it can be thought of as 
a theorem prover. Since it operates by comparing expressions J ike the 
data-base languages of PLANNER and CONNIVER* it can be thought of as a 
pattern-matching language. And sines it includss the lambda calculus. 
It can be thought of as a programming language. 

The following example suggests the sorts of problems addressed by D- 
SCRIPT. A classic problem is that of representing opaque contexts* An 
opaque context is one which does not allow substitution of referential ly 
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equivalent expressions or dose not allow existential quantification. 
For example the verb "want 11 creates an opaque context* 

(1.1) John want* to marry the prettiest girl, 
Thie eentence le ambiguous* It can nsan eitheri 

(1.2) John wants to marry a specific girl who aJso happens to be 
the prettiest* 

or: 

(1.3) John uante to marry whoever ie the prettiest girl, although 
he may not know who that ie. 

Under the firet interpretation we can substitute any phrase which refers 
to the same person for "the prettiest girl". That is, if the prettieet 
girl ie named "Sally Sunshine**, from (1,2) ue can infer: 

(1.4) John wants to marry a specific girl who also happens to be 
named Sally Sunshine* 

Ue cannot make the corresponding inference from (1-3), It will not be 
true that: 

(1.5) John wants to marry whoever Is naned Sally Sunshine, although 
he may not know who that ie> 

Becauee of this property, (1.2) is called the transparent reading of 
(1.1) and (1*3) is called the opaque reading. It le almost always the 
case that sentences having an opaque reading are ambiguous with the 
other reading being transparent. 

Other problems deal with tine reference (compare "The President has been 
married since 1946" with "The Presidant has lived In the Uhite House 
since 1880*') j and representing knowledge about knowledge (e.g., if John 
knows that Bill's phone number Is S87-G543, may ue represent the English 
sentence "John knowe Bill's phons number" as " (KNOUS JOHN (PHONE -NUMBER 
SILL 987-6S43))?), one really wants In one's formalism to be able to 
name a piece of knowledge without having to say what it is. 

For all these types of sentences, D-SCRIPT provides representations 
which allow the correct deductions to be made. Further, it provides 
separate representations for each meaning of the ambiguous sentences, 
and these representations are related in a way that explains the 
ambigul ty. 

A detailed account of O-SCRIPT may be found in trtoore 19731* 
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0. SPEAKING 

The question of "how do -e speak?* has for the most part been ignored by 
natural language researchers in favor of asking "how do we understand? 1 *. 

But there are important questions here about how we organize our 
thought. Understanding how people decide what is appropriate to say — 
applying linguistic knowledge — and knowledge about the state of 
understanding of the recipient — is an intellectual task of no small 
order. 

Uork done to date on such systems as LUNAR, SHRQLU, or other question- 
answering syetees involve rather weak generation schemes or speaking 
components, relying to a large extent on inflexible combinations of 
canned or f i I l-in-the-blank responses. Computer programs have not been 
conscious enough of what they said, making it impossible to carry on a 
proper conversation. In the future, programs in more complex domains, 
such as medical diagnosis, will be required to fluently explain their 
reasoning and to be aware of the knowledge of their audience so as to 
speak at an appropriate level. 

If we want to expose our conversation machine to Turing's test; that is. 
to have a lifelike quality, the system needs a coherent representation 
of the ecendrio in which it is involved. Th« diltdurft* system must be 
designed to exploit this structure. Thus the speaking component of a 
conversation machine, while not critical in some applications, provides 
the glamour that makes conversations seem purposeful, fluent and 
natural. This assumes, of course, the existence of high quality 
listening and thinking components, 

A Talkative Chess Program -- David McDonald 

David McDonald is planning to develop a prototype program cabable of 
discussing its own reasoning processes In natural language. It should 
be able to change its ideas as a result of the conversation. 

Chese has been chosen to be the domain because of the complex but 
straightforward nature of the reasoning involved and because, since It 
It so well contained, there would be no problems with ambiguous 
vocabulary. McDonald is interested both in linguistic analysis of what 
kind of language knowledge is involved in speaking and with the question 
of how that knowledge is structured in a computational environment and 
of what the points of connection are with the other parts of the 
program's cognitive structure. 

Once we have Bf> adequate semantic donain, we can consider for the first 
time <at least in a computational context) what are the differences 
between the choices every language speaker makes continuously. Consider 
these sentences, with the same simple propoei tional content! 
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John** bitching in th« office indicates unhappiness with hie job. 

John's bitching in the office indicates that he is unhappy with 
hie Job- 
John bitches in the office which indicates that he is unhappy with 
his Job. 

John bitches in the office indicating that he "- unhappy with hie 
job. 

What are the differences between these sentences that motivate us to 
picK one over the others? Can we pin down end work with the notions of 
overtone or viewpoint that are involved here? 
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PROJECT 5 
irtPORTANT THEORETICAL TOPICS 

DEFINITION: This "project" brings together a variety of vital questions 
that under! It the problems faced bg all the other tasks. 

Inference from Incomplete Oata 

nodal Logic and Formalization of Common Sense 

Quel i tat We Phytic* 

Scenarios! Frames* and Reasoning by Analogy 

flemory Structures and Difference Networks 

Theories of Semantic Information Retrieval 

MILESTONES* Time scales for recognizable achievements In these 
theoretical areas are hard to eet T but in most cases we expect 
significant reeults within the two years* because of the involvement of 
Ph.D. students- 
Learning flachinesi As always, the limitations on (earning are set by 

what ue know about representing the kinds of things to be learned* 
Ue expect substantial extensions of Suasman's debugging-learning 
system to appear within two years* 

Structural Description: II. Dunlavey's Ph.O. topic concerns extending 

Uinston's "grouping" descriptions so ds to deal with Interact ions, 
e.g.* to understand the structures at the corners of periodic 
brick-like structures — glve.i descriptions of the non-singular 
parts and general knowledge about three-dimensional objects* 

Qualitative Physics: Several members of the Laboratory — Papert* 

Abe I son, Mlnsky, Goldstein, OiSessa. and others ™ are working on 
attempts to formulate commonsenss knowledge about static and 
dynamic mechanics* These will result in several publications 
within the next year or so, flany problems arise in experiments on 
machine intelligence because things obvious to any person are not 
represented in any programs. One can pull with a string, but one 
cannot push with one- One cannot push with a thin wire, either. A 
taut, inextensible cord will break under a very small lateral 
force* Pushing something affects first its speed; only indirectly 
its position! Simple facts like these caused serious problems when 
Charniak attempted to extend Bobrow's "Student" program to more 
realietic applications, and they have not been faced up to until 
now. 

Inductive Inference: R*J. Solomonoff will be a member of the laboratory 
In 1974. Ue hope to see this rssult in some applications of his 
well-known general theory of induction. It is not known whether 
this kind of theory can in fact be applied usefully, even though 
many mathematicians and philosophers are impressed with its clarity 
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i as compared to previous formula* ions. 

Modal Logic: J.R. Geiser will continue to explore relations between the 
AI formulations and those of modern work in modal logic. 

Fra»e Theory and Scenarios* Minsky and several students are working on 
a new strategy for representation of common sense knowledge. An 
extensive publication should appear within a year. 

COSTS* Chiefly in personnel, with console and computational costs for 
heuristic programs in the first three and last areas mentioned above. 
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PROJECT 6 
EXPERT PROBLEM-SOLVING PROGRAMS 



DEFINITION; In this area we place projects in which ideas have reached 
the point of concreteness at which enough theoretical problems are 
understood to justify an experimental heuristic program. The most 
advanced of these Is Chess. R- Greanblatt expects to make important 
advances in incorporating surprise-analysis and some other tactical and 
etretegic elements into his chess program. See the detailed discussion 
In Chess Subproject Section, below* 

Aleo in progress are 

Geometry: A. Broun is working on a Ph.D. thesis In this area. Hie 
program Mill advance our understanding of how to use logical 
methods under the control of knowledge-based methods for 
repreeenting proof and application strategies. The scientific 
directore of the AI Laboratory have taken a strong position against 
the world-wide general optimism about the potential in "wechanica I 
theorem proving" within traditional formal systems of logic or 
modest variants of these. To put it in an almost quixotic form* we 
consider Logic to be little more than a last-resort heuristic to be 
triad wh*n common sense has failed, Nevertheless, ue need to find 
forms of logic that can be used by intelligent systems without the 
restrictions that logicians hav^ traditionally assumed necessary 
and valuable. 

Series Acceleration: R.U. Gosper has discovered a variety of methods 

for decomposing and reassembling convergent mathematical series so 
as to greatly accelerate the rates of convergence over certain 
domains of range and desired precision. He is trying to 
systematize these results. It Is hoped, also, that the theory of 
these methods may illuminate long-standing questions about the 
relative computabl I i ty of classical transcendental numbers. 

Semantic Information Systems: J. Ileidman is beginning a doctoral thesis 
in the area of finding legal case citations, i.e., to present a 
case and have the system find one in the literature that agrees in 
enough legal ly important features* 

Theorem-proving: A. Nevins is extending his logical deduction plus 
heur i stic case-ana I ysi s system. 

fllLESTONESi Projects such as these usually make significant steps in 
tuo years or eo» as they are on the scale of individual large heuristic 
programming projects- Ue expect a definitive publication In each of the 
above in 2 years or less. 
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APPLICATIONS* Ue 
at "checkout" and 
General iy, al I of 
hi th hON to 



do not think of these as application* systems so much 
field-testing of ideas in "tough" environment*, 
the central ideae in these experiments are concerned 
I thi 



Deci s ions under uncertainty 

Decisions under United resource constraints 

Decisions under imperfectly specified goal condit 



ont 



or where one cannot actually check out all possibilities but has to 
choose some action to perform. 



COSTS i 
spec i a 1 
systems 
special 



Personnel, console, and processor costs. None of these require 
hardware* In some cases, the principal researcher will require 
programmer assistance in expediting interactive services, 
displays, or extensions of the high-level languages. 
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PROJECT 6.1 

A COMPUTER CriESS "SUBPBOJECT" 



A, INTRODUCTION 



Although chess research has proceeded at the WT-AI laL and its 
predecessors for almost 15 a ears. it has not previously been made a part 
of project proposals or received explicit funding support. The 
following brief history Is offered to explain the current state of the 
subject. 

How a chess program works 

Both chess programs and humans playing chess (from all evidence) rely 
heavi ly on the method of heuristics! ly I imi ted search . Thie simply 
means that given a position* one proceeds to took at the possibilities 
for one side, the replies for the other, etc. Rapidly, however, one 
rune into the exponential grouth of the tree phenomena and it is 
neceesary to stop the analysis well short of final solution ( Qheckraje ) 
and form a judgment as to the merits of the position. The most common 
way to conceptualize the structure formed In this way is called the a ^m . e 
tree . The given position is visualized as the top node in the tree, 
with alternatives for the side which is to move forming the level 1 
branches. The level 2 branches are possible replies for the other side, 
and so forth. For computers, the module that forms this judgement at 
the terminal nodes is called ths static BflWitlflP ^valuator. The 
"rational" behavior of the players can thsn be modeled by a method 
called mini-na* . This method simply reflects the fact that given a 
number of choices, a player will choose the most favorable to himself, 
and that what is good for the white player is necessarily bad for the 
black player. Using the ■ini-max method, the values assigned by the 
static posi t ton evaluator are successively propogated back to the n-1 
level nodes, n-2 level nodes, etc., until a value is assigned to the 
topmost node. Then the move which led to the the top node getting its 
value is played and the procedure is completed. 

A Few Basic Improvements 

The aj pna-beta algorithm is a method whereby ths tree searching 
procedure described above can be made more efficient. Its basis is that 
once a move has been proved bad (worse than some other move) it is not 
necessary to continue looking to see exactly how bad it is. A plausible 
move generator may be incorporated for the purpose ta»ong others) of 
discarding worthless moves more efficently. A large class of methods 
may be used to attempt to insure that the positions represented by the 
terminal nodes are stable , that is, neither side can greatly improve his 
position in a few moves. However, this remains a very difficult problem 
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(equivalent in the limit to solving chess) 



B. "LEVELS' OF CHESS PROGRAMS 

The chess programs developed In the last 15 years can be roughly 
classified as follows! 



Level -1 

Newell, Simon, and Shaw program and the Kotok et al program, both circa 
13S1. These early programs Mere extremely weak, playing at the level of 
a novice who has played about ten games in hie life* One reason for 
thla weakness was that the programs were overly concerned about "high 
level 4 heuristics at the expense of the baste maxim of chess to "take 
the other guy*s pieces". They failed to play out exchanges* resulting in 
gross blunders. The search parameters were far too narrow for the 
sophistication of the program. For exanple the Kotok program 
Investigated 4 nl ies deep with widths of 4, 3. 2, and 1 (meaning it 
looked at the "best" 4 aovea in the origional position, the "best" 3 
replies to each of them and so on). The program would have no doubt 
played better if it had been set to look only two plies deep, looking at 
8 top noves at each ply. Another major factor was extreme lack of 
debugging and tuning since each program played only a few games in its 
entire existence. 



Level 

The Russian program of 1361, the Greenblatt prograa of 1366, and the 
Carnegie program of 1972 search p| a y pjftla caoture chftini which are 
indefinitely deep, thus avoiding gross blunders that leave pieces 
totally I ogee . They are also characterized by much wider searches than 
the level-1 programs, investigating froo IS to all the legal moves at 
the top two levels at tournament speed settings (2.4 ftinutes per move). 

There is little or no attempt at sophisticated chess knowledge, 
concentrating instead on brute force searching. The programs are 
relatively simple and easily debugged. The early Greenblatt program 
achieved a USEE tournament rating of 14S0, and a Carnegie rating of 
1288* Thie Means that they could beat most aaateur chess players, but 
were In the lowest class of tourna&ent players. 

Level 1 

The Greenblatt prograa of 1368 incorporated various improvements. A 
"simpts" (by later standards anyway) plausible m q v q generator was used. 
It has sone idea of Bftsjt.L9 n . al ■ *V - mechanises for crediting attacking 
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and defending novas, and consideration of discoveries and pfogka^ ea. 
among other things. The qtgtjs position evatuator included naun 
structure as well as material. This program achieved a rating of 1720 
in one tournament and once drew an 1880 player. These are the beet 
achievements of any chess program to date. An 1883 player is right at 
the median of tournament players, and since 280 points corresponds 
approximately to one standard deviation in the distribution, 1720 is 
slightly less than one standard deviation below median. The program 
could beat almost all amateur {non-tournament) players it played. 

Level 2 

The Greenblatt program of 1971 had conceptualization of the idea of 
threat , answering the threat , and other "medium level" concepts. 
ForHfl rfl cutoff is heavily relied on to avoid unnecessary searching. 
Ameer t ion I Ists cal ted wove description tables Mere formed and 
processed, leading to realization of how the various components of the 
move affect each other. For example* if piece X is attacked end piece X 
is also pinned, then the attack assumes special significance. 

In one tournament, the program drew 2 games in G starts, all against 
strong players rated 1S88 or higher. Although this resulted in a 

performance rating louer than that of the I aval 1 program, whan 

allowance is made for the fact the opponents were uniformly stronger* 
the overall performace was only slightly weaker (In achieving the 1720 
performance rating, the level 1 program had the opportunity to beat two 
1408 players). Program complexity is an order of magnitude greater than 
the level one program, and there was a high incidence of bugs. 

Level 2.5 

The current Greenblatt program has further improvements and better 
debugging facilities added to the features of the level 2 program. It 
also incorporates into the static posl tion ft aluator new piece mobi I i tu 
and board control components. It has not been tried in tournament 
competition. In considering level 2 and higher programs, one must keep 

in mind the tremendous accuracy dsmanded of the plausible move 
gener_a.tp c» These programs basically don t look at moves unless they 
can eee they are good. There are many, many reasons a move might be 
good, and it only takes one omission to lose the game. The plausible 
move generator is faced with the task of examining hundreds of moves, 
each of which might be good for between one and two hundred different 
reasons. The actual game tree is vastly smaller than the level and 1 
programs, with a tournament move bassd on making perhaps 250 or less 
moves. The programming effort involved has greatly exceeded origional 
estimates, but it appears the job is very nearly done at the present 

time. 
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Level 3 

The next step is the passing of symbol ic information between levels of 
the search. Thus if certain coves are found to be good at the lower 
levels, information is passed back describing them so that the higher 
level can try to stop them ( surprise analusis ). A feature called 
"threat oass down" propagates information fron the higher levels down. 
Basically, if a move is played with a known threat. It assures that 
analysis is not stopped before that threat is disposed of. These and 
other related features should be implemented in the near future, but 
then a long process of tuning and development will probably occur. Uhat 
rating a level 3 program can achieve is an interesting question. I 
believe an 1880 rating is not unreasonable, but upward progress may soon 
be made impossible without improvements to the end game . in any event, 
this question should be resolved. 

DISCUSSIONj At level 3, at last, we would be developing realistic 
problem-solving skills in a situation where a hostile opponent is trying 
to find and exploit our weaknesses. It remains to be seen just how this 
differs from the "games against nature* that AI programs ordinarily 
consider. To be sure, many heuristic programs have been written to play 
ganes. But we feel that in most cases the approach could be 
characterized as depending mostly gn "trf#-prunlng # concept*, and do (or 
did) not analyse the special features of the competition. (The analyses 
of Newell, Shaw, and Simon are not subject to this criticism, we feel.) 

C. CHESS AS A MICROUORLO FOR A I 

Chess has been of Interest in the past mainly because of Its classical 
nature and certain wel I-publ Iclzed bets, predictions and international 
matches. Recently, however, it is beginning to appear that chess may 
wel I turn out to be an important micro-world for AI in its own right. 
Comparing Chess to other micro-worlds (such as the BLOCKS UORLD), we 
note the following points: 

1) Cheee ie a "real" problem in the sense that is not defined by the AI 

researcher, but by the outside world. Parts of the problem can not 
be defined away without paying a high price, 

2) Chess is a "world" as distinguished from a "mlcro-wor Jd" In that we 

are competing with eerioue adult persons rather than trying to do 
what every six year old can, as in natural language. 

3) Chess offers unique advantages for teaming programs. Note that by 

learning here I dont mean adjustment of pre-existing polynomial 
coefficents or spot assimilation of some fact like an assertion. I 
refer to an extended analysis of a hard problem not specifically 
foreseeable by the programmer and conceptualization and 
incorporation of the results. At a later stage, chess may prove 
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Interesting because of the opportunity it may offer to study ""meta- 
compiling" proceeset. In other words, once the data to be learned 
Is obtained, it must be processed to oore fully incorporate it with 
the existing structure and to increase efficiency. Compare the 
firet tine you ride a bicycle with subsequent attempts, 

4) Chess is already one of the most deeply studied areas of hu«an 

problem solving. In view of the work of DeGroot, Newell and Simon, 
end some others. 



D. PHASES OF THE GAME 

It has been recognized by hunan writers for many years that chess can 
profitably be coneidered to consist of three subheadings, the opening , 
the middle aa&ft and the end?are . It is found that level 8 and above 
programs play reasonably well in the opening using the middle game 
methods, although it is a staple matter to incorporate an opening bpoR 
of pre-determined responses. The main sffort of computer chess 
programmers has been on the middle game, which is dealt with above. No 
serious effort has been made to date to deal with non-trival endgames, 
although it ts frequently said that the difference between a master and 
a expert is endgame play. If the progran has won material In tha middle 
game, it can usually arrange to win by queening passed pawns. 
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PROJECT 7 
IDENTIFYING AREAS OF APPLICATION 



A, INTRODUCTION 



Over the past decade, there has been (in our firm opinion) great 
progress in the field of Artificial Intelligence. A large portion of 

the important work in this area was supported by ARPA's 1PT sponsorship. 
The results are not widely understood, partly because this is a 
genuinely technical domain and pertly because both the press and the 
scientific public have strong, non-technical , prejudices about what is 
possible and what Is not. Thus, In order that the investment be 
effectively exploited, it is important both to produce better 
expositions and documentation of technical progress and to produce 

Informed technical evaluations of the practicality of near-tar* 
appl ication projects* 

Ue are not quite ready to make a conni tasnt to do this. Ue do not think 
there would be any value in commissioning such a study without pre- 
empting the services of one of the very few men who we feel have 
comprehensive overviews. Uhat ue do propose is to work over the next 
year to product a general exposition, based on expanding the plan of our 
1972 Progress Report, which has been demonstrably successful in 
explaining much of the subject to other audiences* 

In particular, we are proposing two small-scale study projects, outlined 
below. The first will be to assess the implications of AI research on 
the general Information Retrieval problem; we feel that progress in 
semantic representations may havs brought this area close to a major 
breakthrough point. The second project will be to develop a work-plan 
for a large-scale assessment of AI potentials) some preliminary sketches 
are presented below. 

B. STUDY PROJECT FOR A PERSONAL ASSISTANT SYSTEM 

DEFINITION: Study project for a major applications the "aanager'e 
assistant". Ue feel that the field of Senantic Information retrieval 

le ready for major advances* Thsrs already exist a wide variety of 
specliized Management Information Systems. They illustrate very well a 

familiar sort of "AI Paradox': 

It l« easier to make a program behave as a highly specialized 
expert than to make one behave in a straightforward, common 
sense way. 

This applies to Information Management just as it does to many other 
fields. Thus, over the years we have seen special systems developed 
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for 

Inventory control 

Classroom scheduling 

Aaeembly-1 ins and job-shop allocation 

Investment trust reports 

Library retrieval 

Accounting data 

and eo forth. One usee such systems Of one usee then at all) on 
conetrained occaeione, for constrained purposes. There are no systems, 
however, that one can use for "ordinary 11 purposes, to do sicple jobs. 

Uhat limite the amount a person can do? Ability, opportunity. 
Information, tine. Every person is a manager, deciding uhat to do 
himself, uhat he can get others to do, uhat information he needs. 

Ue would like to considsr the possibility that the time is ripe for 
developing a "general-purposs" computer based Assistant system. Ite 
purpose would be to save one's time, help get and keep information, and 
perform choree that either one has to do or one does not do because of 
all those limitations. To present the image, here Is a scenario 
involving a Professor and a Personal Assistant program, 

Thle example is chosen only because ue know this situation best. Any 
reader more familiar with such functions as Administration, management. 
Engineering Design, Military Command, etc., can easily construct a 
similar ecenarlo for that kind of function. 

A professor happens to read a paper on plasma stability by a 
certain author. He does not consider the main result very 
original or important, but a certain example in the paper might 
be relevant to one of his students* work. 

He tells the Assistant Program the citation of the paper, and 
entere hie remarks. In 1975, he would have to type it int In 
1980 there is a good chancs he could speak directly to the 
machine. 

The Semantic Analyser sends a note to the student (or to the 
student's Assistant Program!. It makes a note about the 
citation and the Professor's remarks in a data bank for 
information retrieval about Plasma theory and related subjects; 
It assigns a description that will not attract attention unless 
there is some reason to suppose that the situation might have 
changed regarding the paper's originality. 

Some months later, the Professor needs the example! He cannot 
remember the citation or the author's name. He does not even 
remember that the paper was about plasmas — the example 
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concerned classical Hamiltonian machanics. Fortunately, he does 
remember that he had thought the student night be interested, 
and that the incident occured last Uinter, He tells the 
Assistant Program this, and it instantly produces the citation, 
and displays the relevant part of the text on the display* 

In the meantlne, it so happens that the author is announced to 
present a Physics Colloquium at Harvard. The Assistant Program, 
uhich reads the daily newsletters every norning, notices the 
name and leaves a T1AIL" signal for the student Mho, it thinks, 
might possibly be interested in meeting the visitor. 

The message can just as easily tie sent, over a computer network, 
to any known individual who might want such notification and has 
agreed to receive such services. 

In our study so far, we have come to the conclusion that euch a system 
could be brought to a practical level — for use by a computationally 
sophisticated person — - in two years. It would take longer, perhaps 
another two years, to extend its capabilities an<i smooth out the human 
engineering to make it equally ussful to non-specialists. There are all 
sorts of hard problems that make precise time estimates difficult, since 
so rauch will depend on concurrent progress In the computer manipulation 
of common-sense ideas and meanings. In any case, we believe that the 
possible payoff of such systems could be enormous, and have all kinds of 
appl \ cat ions- 

MILESTONE: Ue are NOT proposing to embark immediately on such a 
project* Instead we ^r% planning a serious feasibility and exploratory 
study during the coming year. The result should be a report surveying 
the state of knowledge, comparing strategies of attack, determining 
which applications (Administration? Managemsnt? Personnel Supervision?) 
would be best for the first attempt at building a working system. 

PERSONNEL: Ue do not feel that our present staff has ail the expertise 
neceseary to make such a study, and ue propose to use a panel of 
consul tante. Involving such people as Engelbart and Teitelman, who have 
made studies into different aspects of this area. 

C. STUDY PROJECT FOR ASSESSING AI APPLICATIONS IN GENERAL 

DEFINITION: A thorough, substantial assessment of the current state of 
knowledge about machine intelligence; the important solved and unsolved 
problems, and assessment of those areas in which applications seem most 
likely to pay off in the nn^r and middle future, 

MILESTONE: A book-length discussion of the issues, of clarity and 
composition suitable for non-special ists, to be completed by the end of 
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1975. 

PERSONNEL; flarvin Minsky, Seymour Papert, possibly others. 

Some general remarks on the state of AI Ml 1 1 be found in Section 3 of 
this proposal. Immediately below are some areas we feel competent to 
organize more detailed assessments for* 

0. HAND-EYE MANIPULATORS 

It is hard to think of any field that could not be transformed by the 
availability of machines that perform physical actions quickly and 
carefully under visual and tactile feedback control. The projects 
proposed here are especially relevant to MAINTENANCE, INSPECTION and 
REPAIR of complex systems, by assisting relatively unski Med personnel. 

In particular, the attention of this Laboratory will be concentrated, 
whenever the extra complication is not excessive, on the development of 
methods and systems to deal with "Micro-Automation" — that is, physical 
manipulation of very small components such as will be found in advanced 
electronic systems. This is an area In which there has been 
particularly little advanced development, limiting the practical 
appl icat ions. 

This area Is of particular interest because human workers do not compete 
in it very well, AI techniques in this area will also be useful on the 
Macro-Scale, e.g*. In C0NSTRXTI0N situations where there are problems 
about human skill, manpower, safety, or speed. 

Eventually we expect to see these devices employed in areas of critical 
civilian concern, e.g. MINING, UNOERSEA DEVELOPMENT, and SPACE. 

For example, unless fusion power becomes aval lab 
more MINING and/or SOLAR POWER. Advanced Automat 
low-grade shale mining or for the construction a 
maintenance of enormous arrays of solar cells on 
■lining, AI techniques might help solve problems 

Finally, we expect to see major applications in 
two decades. 

Some of these are discussed further in our MICRO-AUTOMATION proposal, 
MIT-AI memo 251, 

NOTEi In earlier proposals and publication* we observed that U.S. 

"knowhow 1 * in advanced automation has been permitted to coast along, Ue 
believe that the skills involved in uhat we call "advanced automation" 
uill become vital to many of the other near-crisis problems of the near 

future, as noted in other items below, Ue recognize that ARPA is not 
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the agency to carry the research load for this area, but ue want to put 
on record that Me expect the work described In this proposal to have 
very substantial contributions to those areas. 

The problem of low U.S. productivity is nou recognized* both with 
respect to foreign competition and in absolute internal matters* 
Certain Kinds of advanced automation can forestall or at least space out 

the adjustments that might otherwise arrive in destructive waves. Ue 
cannot take up here the question of social consequences of advanced 
automation, but will state our position briefly. Certainly, if advanced 
automation were thoughtlessly applied to existing Industries without 
compensatory planning, serious dislocations could ensue; this 
reeponeibi 1 1 ty of economic planners must be recognized. On the other 
hand, ordinary marketplace adjustments do not seem able to handle rapid 

large-scale changes in such areas as agriculture, energy, housing, 

transportation, education and health services. Only advanced automation 
can make possible large-scale physical adjustments without similarly 

large-scale social transients* 

E. AOVANCEO INFORMATION SYSTErtS 

Ue believe that AI research can show the way to computer-based 
information systems far more capable than have ever been available. Ue 
plan to attack the area of Information Retrieval, both for traditional 
data-base and library problems and for aore personal ttANAGEttENT 
INFORMATION SYSTErtS problems. 

The new Information Systems should help to Increase the effectiveness of 
individuals who are responsible for complicated administrative 
structures, as well as for complex information problems of technical 
kinds. In particular, the services will be available and designed to be 
useable over the ARPANET, and will be designed to interact with the 
personal systems of other individuals to recognize conflicts, and 
arrange communication about coupon concerns. 

F. AUTOMATIC PROGRAmiNG, AUTOrtATIC DEBUGGING 

In all the areas noted above, and already in many conventional areas, 
computer-programming plays a large, expensive and frustrating role. Not 
only expense, but delay and unreliability of programs threaten to grow 
rapidly in the future. By pushing ahead on the related fronts of 
Automatic Programming, Automatic Debugging, and "SELF-OOCUtlENTlNG" 
programs we believe that this trend can be held in control, 

G. flEDICINE 
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There is now more real physiological, pathological, and therapeutic 
knowledge than any small group of physicians can encompass* But there 
Bre not enough "specialists* 1 to eerve as consultants and this problem 
will become steadily worse, to the point that present inadequacies and 

inequities wi 1 1 become dangerous- In another project, ue plan to apply 
our knowledge-based programming technology to the construction of a 

"computer consultant" — a large heuristic progran that will be able to 
provide to general medicine the equivalent of a highly skilled 
consultant-specialist — with a level of competence well above that 
available in the average non-research hospital service ar&a* (The first 
euch system will, probably, be concerned with renal and cardiac 
problems. ) 

The resulting services will be available to unspeciai ized personnel In 
rural or otherwise remote and inaccessible areas, over computer network 
connect ione. 

Ue are asking for N1H support for joint work between AI workers and an 
outstanding teaching hospital In this area* 

Similar health information services mi J J have to be developed for other 
medical areas, and for diagnosis, treatment planning, special therapiee, 
and heal th services administration. 

There are direct applications of A! results, both from the robotics area 
and from the general information managing areas, for dsvices for 
PROSTHETICS for the handicapped. Gsnsrally, all such work has equal 
value as potential -augmentation* of normal persons. 

H. EDUCATION 

There &re already significant applications of AI ideas to training and 
education, far bsyond the "first-order" services provided (for better or 
for worse) by the earliest "computer-aided instruction" systens. The 
value of the ideas in our <NSF-supported) LOGO project are widely 
recognized by educators. In attempting to understand how intelligence 
develops, it is of course especially fruitful to concentrate research on 
younger children for scisntiflc reasons. (This work is supported by the 
NSF.) 

However, we have evidence that the methods developed in our educational 
research arQ equally effective for use with adults, and we expect they 
will help solve problems of equipping people with skills adequate for 
useful occupations even if they are from backgrounds considered 
handicaps today. 

In any case, there is a close relation between the development of AI 
theories and the work on huaan development in our group. 
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I. OTHER AREAS OF APPLICATION 

LANGUAGE and SPEECH research will make the new systems available to non- 
speeialiete, as well as to people with busy hands! 

LARGE SYHBOLIC COMPUTATIONS such as are done Hlth MATHLAB. will make 
practical new areas of physics and engineering. 

PICTURE PROCESSING. PATTERN RECOGNITION are of continuous concern In Al, 
and should continue to be productive areas. 

Several of these were further explained in our flicro-Autornat ion 
proposa I * 
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SECTION 3 
RESEARCH ON ARTIFICIAL INTELLIGENCE 



A, INTRODUCTION 



This section discusses some aspects of the state of the art in 
AI t that relate to the projects of this proposal. It is not a 
general discussion. In fact, as noted in (2.7J, we are 
proposing to create just such a general survey and assessment; 
there it nothing in the literature today adequate to give a non- 
specialist a competent overview of what has happened in the past 
five years. 

B, TIIIE^SCALE OF APPLICATIONS 
TO CRITICAL AREAS OF NATIONAL CONCERN 

In each of a number of important fields, soms of uhich are discussed in 
Project 7, conventional technology is today pushing against one or 
another bottleneck involving effective use of information, knowledge, or 
real-world Interactive capability. Ue believe that the uork proposed 
herein u i 1 1 play a vital role in each of these areas* 

Our position is that AI can help with many Kinds of complex programs — 
because of its new ways to aanage — or at least, describe technically 
and formally — kinds of interactions that in the past could be treated 
only empirically and informally. 

Ue believe that many of these areas can be open to our techniques within 
five years — some sooner and some later. In some cases it is clear 
uhat must be done and how long it should take. In others we have to 
speculate, where the problems B^e not entirely understood in case there 
may be serious unforeseen difficulties. A lot depends on how urgent 
these areas &rm seen as by the relevant agencies- And a lot depends on 
the availability of talented young workers in this field* 

In any case, If ue start on the important problems now, 
implementation of important, large-scale applications could 
bsgln In less than five years, given appropriate priorities. In 
section 2 we proposed "milestones'* that suggest how certain of 
these areas can be developed. 

IMPORTANT! Ue are not saying that the large-scale aplications will be 
in operation five years from now! Ue mean that by that time it will 
become a matter of investment and priorities. Right now, there simply 
would be no way to make 508 people work on implementation of an 
intellgent management system; the senantic structure has to be drafted 
first. There is no way to get a lot of people to work on an Automatic 
Programming and Debugging systen; the representation of program 
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intentions and seheaa has to be designed first. There is no way to get 
a lot of people to work on a big natural language systemt we first have 

to specify representation of scenarios, time and tense, etc*, etc.* etc. 

(Ue could right now, ask 58B people to help design the mechanical sida 
of Industrial robot systems, because we have a decade of complaints 
about inadequate mechanical devices, without good force or touch 

sensors, and marginal visual hardware- J 

In five years, we would expect that enough testing of different 
representational schemes uill have been accumulated to make such 
deci ei one comfortable. 

Why do we say "five years"? The development of useful MACHINE VISION, 
along the lines we proposed around 1965. is just maturing* Its first 
useful applications in assembly of unpositioned components Are just now 
in progress in several laboratories. Ihe MATHLA8 project, also dating 
from that period in our Lboratory. is just now serving its first outside 
users. The current applications of NATURAL LANGUAGE processors that are 
Just In the past year beginning to use semantic structures are in a 
large degree results of work of ours and others that first took form, 
again, in the middle '60s. It might be noted that the whole area called 
Artificial Intelligence originated, roughly, around 1955, in the sense 
that it could be distinguished from •cybernetics" and "control theory** 
and "operations research* by its amphasis on symbolic description and 
related processss. If the past is any guide, we see that the first of 
two decades explored many theoretical avenues. The second decade 
developed some of the better ideas to the point where applications could 
be considered. There are still many major difficulties in all areas; 
besides theoretical problems, it is still very hard to write and debug 
the large systems involved, and they run slowly and expensively on 
present hardware in the current implementations of the languages. 

In this perspective, 18 years might seen more plausible for development 
of large-scale practical applications. But the aembers of this project, 
which has made rapid advances recently, expect the field to move even 

faster than in 1963-73, This Is partly because of recent theoretical 
progress, but also because: 

In 1963 there were only perhaps 20 full time workers in Al — that 
is, heuristic programming and symbolic representation. There 
are now at least 280. Host major universities have AI courses 
now; in 1363 there were only 4 or 5* 

The problems seem much more urgent in this decade, although the 
relevance of AI is not yet widely recognized* See the 
specific items below In this section. 

The needed large-meaory, interactive, computat ional facl I 1 ties arc 
now generally obtainable. In the cosing decade the reductions 
in cost and the availability of Network Services will make 
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them widely available 



Technical tools are improving fast enough to have an effect 
Automatic Programming aids of the sort being developed 
r?_311 nnnlrt h* J n in nthwr AT nrninrtA in thrift rsr fmii 



«OOn, 
(see 
C2.33) could help in other Al projects in three or four years. 



C. IHPORTANT PROBLEMS IN AI 

Artificial Intelligence, a* a new technology, is in an intermediate 
stage of development* In the first stages of a new field, things have 
to be simplified so that one can isolate and study the elementary 
phenomena. In most successful applications, we use a strategy we call 
"working within a riicro-Uorid** 

In the rest of this Section, us discuss a variety of problems that arise 
when on* develops a microworld to prove out some idea and then wishes to 
expand the system toward real-life applications. 

[licro-Uor Ids 

Over a long period, ue have learned now ideas about intelligence could 
be tested and developed, and a style of research strategy has emerged* 
The selection of test environments is* ue think, very critical, and many 
blind ends and traps have swallowed up workers who had not given the 
question sufficient thought. For example, the use of two-dimensional 
patterns for "vision research" led many groups away from discovering 
important principles about scene-analysis — because the basically 
reversible transformat ions of a two-dimensional visual world do not 
require one to use symbolic descriptions. Unfor tundtely, this leads one 
— several steps later — away from adequate theories for learning! 

A good example of a suitably designed Micro-world is shown In the well- 
known project of Uinograd, which made many practical and theoretical 
contributions to Understanding Natural Language. Much of that system's 
power cones from the way in which the semantic system is able to 
represent things that nappen in the so-called Blocks World. That test 
environment contained just the right order of complexity for the level 
of semantic competence that seemed achievable at the time. Uinograd'e 
nicro-Uorld contained essentially onJyi 

a few kinds of (physical) OBJECTS. Each object had 

only a few PROPERTIES (color, size, location). Also, there 

were 
only a few ACTIONS (grasp, lift, move) available. 

The interactions between these were relatively manageable — the 
"problem-solver" contained in the system was "state-of-the-art" — eo 
that things seemed more or less within the comprehension of the 
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exper 1 menta I I anguage-seaant ic sys ten. 

Problems in Expanding a Microuorld 

Sine* the Uinograd demonstration and thesis, several workers have been 
adding new elements, relations, and features to that system. That work 
has not gone very far, however, because the details of implementation of 
the original system were quite complex and, accordingly, it took quite a 
long time for subsequent workers to analyse, document, and systematize 
the system for use by others. (It is only now taking acceptable form.) 
The work of Uoods (at BBN) is currently In better for* for other 
applications; this is partly because its structure was from the start 
more uniform and systematic, demonstrating (we think) Just the kinds of 
trade-offs being discusssd. In the work of Uinston's group on the 
"visual" Blocks Uorld (the extension of the original Hicrotlorld, which 
was the "Polybrick* system of Guzman's thesis) progress toward including 
more and more has been steady* 

The Need for a Knowledge-Structure Theory 

Obviously intelligence Is not merely having knowledge. One needs good 
ways to decide what is relevant to a situation — Intelligent 
Information Retrieval* Uithtn the nicroworlds that have been studied in 
AI research, many problems and proposals have been examined, and these 
have led to some powerful techniques. At first, many workers hoped it 
would be possible to separate the problems of reasoning and deduction 
Intoi 

A basis of factual knowledge 

A (possibly complicated) procedure for using the knowledge 

But the separation leads to serious problems. Too much of one's 
knowledge is concerned precisely with which (other) knowledge should be 
applied to various kinds of situations. Unfortunately, this turned out 
to be harder to "represent" than did "ordinary* factual knowledge. 

Ue note that at the MIT Al laboratory, this problem is taken much more 
seriously than at most other research centers; this is why we do 
relatively more work on designing advanced knowledge-based systems and 
Jess work on trying to extend the "logical theorem-proving programs", 
Uhlle we agree it is of some importance to find out how far such systems 
can be made to go, we don't think they will ever go far enough to solve 
most i mportant pract ical problems. 

[n particular, we think that some extremely Important kinds of knowledge 
(vital to solving any difficult problem) were overlooked almost 
entirely, In the early days. These are the "facts of life" about 
interactions, bugs, bottlenecks, etc.. which every child comes to know. 
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Ue believe that now that the area it recognized as a manageable 
technical subject (see IS. 31) there will be a large change in the 
capability to apply larger bases of information to hard problems, Most 
important, we think, in the next year or two is to get more work done on 
understanding "types of bugs" — as It is called in Sussman's thesis — 
and apply these ideas to automatic programming and debugging 
appi icat ions. 

Representations 

It le generally agreed, also, that success in problen solving depends on 
finding a good way to represent one's knowledge so that it can be made 
to fit the problems one is trying to solve. That this is true is shown 
rather clearly in the history of nathemat ics, in which "mere notational" 
issues aade very large differences in practical effects, viz., the power 
of the differentia) calculus notations, or the modern vector 
representations. 

Nevertheless, although the importance of adequate representations is now 
generally recognized as central to AI, we think it is important to 
remember that the classification and isolation of Knowledge is not yet 
adequate. The recognition that there are »3ny "types of bugs" — that 
is, that there are different kinds of bad interactions between simple 
processes ™ which require different kinds of interventions, was not so 
rauch a matter of representation as of rscognition of the problem. 

Combining flicroWor Ids 

PeopJe ask: now that you have programs that do "this", and ones that do 
"that" — why don't you combine them all to get one program that is much 
more intel I igent? 

Ue cannot usually do this. Sometimes the difficulty le sinply that the 
programs use different representations ~ they can't communicate in any 
one language. The cure for this is usually to try to rewrite both. 
Another approach, in principle, is "modularity*! Make all your prograws 
out of compatible modules. In electronics, this is pretty easy; maKe 
sure all inputs and outputs are "TTL-level compatible; make all power 
supplies run at 5 or 15 volts* 

The trouble is that the Idea of modularity, itself, is not very 
appropriate to intelligent systems, in which the problems of interaction 
are the important and difficult ones. Uhen two "knowledge-modules" &re 
connected, one has to put — somewhere — a key to the most likely 
interactive types of bugs that can be expected to emerge. 

It regains to be seen whether such uniform approaches as Hewitt's ACTOR 
module (see 15-3] > will yield substantial gains here, or the Predicate 
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Calculus formulations of McCarthy et al. at Stanford, Ue are quite 
confidant that there are Important Immediate advance* to be obtained 
from the self-annotating sel f-debugging approaches proposed in CS.31, 
over the next few years, and that these have immediate as well as long- 
term appl i cat Ions. 

The Blocks Uorld — Success in Combining Two Hicro-worlds 

By something of a coincidence — certainly for quite different reasons - 
- our research on Vision over the past few years had also converged on a 
P1icro-uortd of the very sane sorts of simple physical geometrical 
objects and actions, and ue use the same "Blocks Uorld" caption for the 
experimental vision environasnt that uas used for that work, Ue note in 
passing that many visitors and readern missed the point rather badly, 
and were skeptical that anything very realistic could be learned by 
working with perfect simple shapes, no textures, no noise, no complex 
curves or "soft* surfaces. But we claim that through this careful, 
sometimes tedious analysis we laid the foundation for the main lines of 
the successful systems now being developed and demonstrated! certainly 
at the higher levels of scene-analysis (where all contemporary groups 
are now following our general outline) and (though to a lesser extent) 
even on systems that deal with poor, noisy, textured situations. 
Basically, we took the position that 

"One should not directly attack the problen of recognizing a 
pattern immersed in noise, until one is able effectively to 
recognize it in the clear*. 

This may seem to be common ssnse but experience shows that moet 
beginners feel it too simpleainded to take seriously. 

He note in passing that our own recognized progress in non-trivial 
machine-learning is In large part due to a related principle; as 
McCarthy has put it, 

...one should not attempt to make a machine learn something 
unless one is sure there is some uay to tell it that thing ■»« 

i.e., that the machine has access to Bn internal representation that can 
in some natural way encode ths desired infornation. Uithout adequate 
representation one can do little, with it, other difficult problems 
often melt away. Thus, in U ins ton's thesis, once we had an adequate 
representation for DIFFERENCES between other descriptions, several 
interesting kinds of learning became easier to program, and some 
reasoning-by-analogy was easy to incorporate. 

On the other side, the same principle has been used to show that one 
very popular approach to building learning machines was inherently 
defective. The core of our resuJte in the Theory of Perceptrons was 
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based on separating the questions about learning from those about the 
machines' representational capabilities; then* when the latter were 
shown to be inadequate, we were able to prove (PERCEPTRONS, HIT Press, 
1989) that no series of 'training sessions", however prolonged, could 
make the poor machine learn its lessons. And, as a by-product of that 
analysis, we did discover a nuaber of limited, but interesting and 
useful tasks that perceptron machines could in fact be oade to do. 

Uncertainty 

There are a number of areas in which present foundations are not quite 
strong enough to support the proposed applied projects. These must be 
studied further, in smaller nicro-wor Ids. Us have to get better control 
of the areas briefly discussad below in order to bring the applications 
fro» the "control led-*environnent demonstration" phase to the real 
application requirements. 

Types of Uncertainty 

As AI programs get more aebitious in ths size and scope of their 
knowledge bases, we will encounter eore and more situations in which 
conclusions are unsure. This is not a nsw problems uncertainty is 
familiar even in the most highly defined situations with no chance 
element. For example, in Chess, one must "take chances". 

Taking chances does not, housver, mean using probabilities! The best 
game-playing programs have not used probability models in their 
performance. (Uhether they use such theories in their conception and 
construction is another matter entirely, and one that is hard to settle 
or evaluate. Thus, in Samuel's Checkers programs, ons can argue both 
sides of uhether probability is involved. Certainly, a close relative 
of Mathematical correlation is used.) In anu case, ths new areas will be 
necessarily engaged in more 'plausible inference" and eviaence-gathering 
activities. In the course of that research, ue can expect to face and 
develop ideas about decision under uncertainty! we predict that the 
results will be quite different from classical decision theory and 
classical utility theories. On the other hand the results must also be 
practically effective. It will be interesting and, we hope, valuable* 
to see how these theories will relate to the traditional decision- 
theory, game-theory, and economic-theory models. 

Similar problems appear in more iauediately important areas. In 
Automatic Debugging, one has to face the problen of plausible inference 

of intentions and plans in a program — see [Appendix 5,3.1 — unless 

the programmer has already spelled this all out. 

In any case, there ar& other arsas of uncertainty. The quality of 
evidence depends on its source. Ons nssds policies and methods of 
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accounting for authority, reliability* and strength of conviction. 

In Chess, one uncertainty comes not so much from lack of information but 
froai too muchj the impossibly large full search tree la too large to 
examine and one must adopt some strategy that uses less information. 
Note that the uncertainty, in Chess, is in part that one does not know 
the opponent's strategy. To cope with that kind of situation one does 
beet to build some sort of MODEL of the opponent (or t more generally, of 
the environment one is in). But different problems, in such a fix, 
depend on different ways in which one set up that model , and one should 
make provisions for noticing at least some such dependencies. I4e 
believe that the new program-accessible comment schemes like those in 
Sussman and Goldstein's theses, will show ways to do such things. 

Qua) i tat i ve-quantat ive i esues 

In controlled demonstrations we have found that ue can usually sidestep 
problems of estimation and uncertainty by a variety of heuristic 
devices. There is some reason to believe that this can be done quite 
generally and* frankly, most of the project scientists do not bslieve 

that humans ordinarily use (in their heads) the kinds of probabi I let ic 
decision and utility models that have occupied the primary attention of 
most "quantitative social scientists". Certainly, ue still feel this 

way about the use of "analog" models for visual imagery — another area 

in uhich the majority of psychologists think "quantitative" machinery 
must be required. Ue believe this is a mistake based on unfami I iar i ty 
with the power of "symbolic-descriptive" mechanisms that have evolved in 
work on AI, The same may be true in decision theory, and the problem ie 
beginning to face us as ue begin to deal with much larger varieties of 
kinds of information within the sans system, Ue expect, for example, 

that the work on Chose, which features the program* s necessary ignorance 
of the opponent's plan, will clarify this issue, and our planned work on 

the Management Assistant project, which also will range over a highly 

inhomogeneous data base, will also have to face such issues. 

There are many other ways in uhich qualitative issues enter common sense 
situations. Everyone knows ways to use qualitative quantities; if A Ie 
very near B and B is very near C then A is (at least) near C. But one 
can also say that A is very near C under usual conditions. Obviously 
one cannot iterate this very many times* 

Such issues are implicit In many of the current projects. They are also 
important in the Medical Assistant project area (which we expect NIH to 
support) that also concern some of the same Laboratory staff oembers. 
Ue expect a substantial interchange of efforts in this qualitative- 
quantitative representation area between the workers of this proposal 
and uith our colleagues U. Martin. A. Gorry, U. Schwartz, and othere In 
the proposed NIH project. 
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Extended Events, Frames, Scenarios 

Ue believe that the problem of representing real world knowledge ui II 
have to be faced by dealing with such larger chunks than our colleagues 
have believed necessary. U!e feel that the problems that beset the 
"theorem-proving" projects and. generally, all those using methods 
related to the formalisms of traditional "Mathematical Logic" come from 
the limited resources such systems have for representing the 
Interactions within highly structured real systems. Our knowledge about 
the real world, or about those subjects in which we are particularly 
competent, is not a bland, uniform structure of simply-interconnected 
"facts"* Ue envision it as more like real geography; houses are not 
equally near one another, but are arranged — for good but intricate 
reasons — in towns, cities, netropoli; these have centers and capitols 
of many different sorts* The road system reflects this structure more 
or less perfectly. Similarly (though the simile eoon becomes fatuous) 
our knowledge is made up not of similar knowledge about many different 
things, but of elaborate constructions about a few things* Plus — and 
this is vital — higher level data about how other less intimately 
familiar systems ^rm similar and how they are different* Thus* the main 
tool of reasoning — in the opinion of the principals of our project — 
is not regular deductive logic but, rather, procedures for drawing 
analogies and for making plans to carry out the'- suggestions. 

The elements of this kind of knowledge right then resemble stories 
rather than axioms; scenarios rather than snapshots, typical -examples-* 
plus-advice about applications rather than "general principles 11 . Our 
first attempts to use these ideas will be in the areas of Natural 
Language and Scene Analysis, but the general idea is already influencing 
plans for the other applications projects. 

Heterarchical Pr ogramming 

Programming and debugging are getting too hard. This is because the 
systems are getting larger and more complex. They use, in a single 
system, many different kinds of information, and the interfaces which 
translate between one representation and another add to the complexity 
of the system. Fortunately (1) better understanding of heterarchical 
systems is rapidly growing. (2) Ue frankly expect that our recent 
progress on Automatic Debugging will help — perhaps not directly on the 
programs themselves -- but certainly on the new style of internal 
documentation suggested by the phrase "program-accessible representation 
of programs' INTENTIONS". 

Goati Programs with "common sense". One should be able to ask a 
program why it did something, how it works, why it said something. 
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To answer such questions, the program has to contain a representation of 
its own activity « in terns of ' intent Ions' or higher-level goal- 
oriented comments, rather than or in addition to the usual "declarative" 
or "procedural - program- language description of the process to be 
carried out. 

Indeed, we believe that such an "explanation*, or "excuse" language is 
neceeaary not only for making sense of a program's activity and 

operation for intelligent application, debugging, and extension -- 

but also for developing any large prograa in the first place. But 
usually, at least today, the goal oriented description exists only 
informally in the programmer's mind. 

In Sussnan's thesis we see sons steps toward this. Uhen a test program 
shows a "bug", a higher level program tries to explain it, perhaps in 
terms of side*effects of conflicting actions, or side effects of 
competing goals. If, for exanple, the requirements for concurrent goals 
interact, so that fulfilling the conditions for acheving Goal A make it 
impossible to achieve Goal B, it may suffice simply to achieve goal B 
first. But the program will only think of doing this If it can explain 
the failure in terms that suggest that those goals are in fact closely 
associated and therefore candidates for swapping. 
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SECTION 4 
THE ARTIFICIAL INTELLIGENCE LABORATORY 

A. INTRODUCTION 

The rtIT Al Laboratory has evolved from a snail group of students working 
with John McCarthy and Marvin Minsky in 1958. It was originally part of 
flIT f s Research Laboratory of Electronics and the fllT Computation Center. 
It joined in the creation of Project MAC as the Artificial Intelligence 
Group, and became a separate I11T Laboratory some years later. During 
the period of ARPA support* it has aeen co-directed by Marvin Minsky and 
Seymour Paper t. 

In this section we describe some of its activities accompl ishments and 
problems. 

B. SYMBOLIC APPLIED MATHEMATICS 

The first serious project on getting a computer to work with literal 
formulas and real Mathematical principles rather than numerical 
calculations was done here by Slagle, whose 19S1 Ph.D. Thesis showed how 
a heuristic program could compare favorably In performance with a 
bet ter-than-average MIT first-year student on symbolic Integration. The 
program could not deal at all with the context of such problems, so that 
it was useful only once the problem had been put into the right form. 

This demonstrated that heuristic techniques could deal with symbolic 
mathematics in a rather natural, lifelike manner. The behavior of 
Slagle's program resembles rather closely that of mathematics students 
who are very intelligent but not particularly expert in that area* 

Some years later the same problem — symbolic indefinite integration -- 
was attacked by Joel Moses. His thesis exhibits highly expert 
performance over a much wider range than did Slagle's. Moses* program 
demonstrated publicly that we were at the stage where AI Methods could 
produce a mathematical "assistant 11 that could really extend the 
capability of a serious user. 

At the same ti«e, U. Martin addressed himself to other problems in the 
area of symbolic applied mathematics; to making theories of 
representation, interaction, simp I i f '.cat ion. and so forth. The success 
of the two theses of Martin and Moses suggested joining forces, which 
led to the PlACSYflA system — or the MIT MATHLAB project — that Is now 
an independent part of Project MAC. 

Since then, many other centere have attacked parts of the problem, and 
this field is just now becoming a significant addition to the tools of 
the scientific world in attacking large, semi-quantitative problems in 
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applied mathematlce, engineering, and physics. 

Another, parallel development In thlt period has been the uork of Bobrou 
and Charniak, Bobrou developed a heuristic program, STUDENT, that 
worked on lees synbolic, more realistic "word problems" at the high- 
school algebra level. In thie uork, the emphasis uas not so much on 
format mathematics ae on semantic* and natural language- Bobrou shoued 
that by using a semantic model of uhat sentences (probably) mean, one 
can sidestep issues in linguistic theories that seemed very serious if 
attacked without reference to meanings- The success of this prototype 
had a noticeable effect over the next feu years in redirecting the 
attention of computational linguistics to the practicability (and, ue 
think* the indispensibi I i ty) of involving neaning uith syntactic 
anal ysi e. 

Years later, E. Charniak produced a Master's Thesis in uhich some uord 
problems in Calculus could be handled. This thssis raised n%u Issues; 
It became dear that as Charniak attenptsd to include in his scope such 
issues as time-analysis, simple mechanical statistics and dynamics, and 
properties of conmon materials (you can pull uith a rope, but not 
push I ) , the earlier Micro-uorld of simultaneous linear equations and 
phenonena that had sufficed for Bobrou's problems uas not adequate. 
Indeed, the extension of symbolic applied mathematics to non-symbolic 
real-uorld problems still awaits refinement of conmon sense knouledge- 
etructures. Ue hope to fill this gap as a result of uork on 
"qualitative physics** (see S.6K 

C. VISION 

There is too little space h%rn for an adequate summary of the AI 
Laboratory's uork on real-uorld vision. There ui 1 1 appear a survey in P 
Winston's paper in the August 1973 Proceedings of the International 
Conference on AI, Stanford, 1973, 

Briefly, ue claim that it uas primarily our line of emphasis that has 
created the contemporary atmosphere of optimism and accomplishment in 
Hachina Vision, The main points uerei 

Emphasis on Symbolic description rather than Picture-Transformation 

Emphasis on Hsterarchical use of real-uorld knouledge 

Emphasis on problems of 3-D occlusion and figure-ground separation 

This uork resulted in several Internationally knoun papers* notably the 
Ph,D theses of A, Guzman, P. Uinston, and D- Ualtz, 

The "lou-level" problen of finding real physical features In picture 
scenes has been a very difficult ones important contributions u%rm made 
in uork by J. HoMouay, R. Greenblatt, A. Griffith. T. Binford, B. Horn* 
and A. Herskovits in our Laboratory. Three Ph.O theses on other Vision 
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topic* were written by A. Griffith. 9- Horn and l» Krakauer, exploring 
different mathematical and structural models of such topics as shading, 
contrast, illumination gradients, highlights, focus maps, etc* 

The outcome of al I this was the modern approach to vision generally 

called "Scene-Analysis* — a tere that serves mainly to distinguish the 

approach from its predecessor, usually called "visual pattern- 
recogni t ion"* 

0, NATURAL LANGUAGE 

In our Laboratory, the work on Natural Language developed in the context 
of understanding common sense semantics rather than from the milieu of 
classical linguistics. The work of Bobrow, noted above, and the work of 
B. Raphael on a question-answering program that used contextual clues to 
solve problems of semantic ambiguity uere the first attempts to take 
this approach. The work of Charnlak, as noted above, was a next step, 
but showed the need for a more carefully developed llicro-world of 
context. In Uinograd's thesis we saw these and many other ideas from 
linguistic theory brought together and applied to a more adequate 
symbolic Blocks Uorld that resemblss (superficially) the one used in 
our early vision research, Most important, perhaps, were Uinograd's 
innovations In making the syntactic-semantic interactions actually work 
in a heterarchical programming systems a dream that had never been 
realized effectively because of inadequate technical understanding (and 
courage) . 

Semantic Information Processing is now one of the most active fields of 
AI, and promises to yield systems useful in many practical areas. 

E. ROBOTICS 

In the history of efforts to understand how to make autonomous physical 
assistants, the AI Laboratory claims a very special role. The first 
such system, so far as we know, was the tactile-sense heuristic program 
of H. Ernst which, in 13G1 was able to search for different objects on a 
table and assemble them into a tower or put them into a box. Some years 
elapsed, in which we made plans to move in the direction of visually- 
controlled manipulators, and final ly» in the mid-63 f s R. Gosper, using 
programs based on earlier work by J. Hoi lousy. R« Greenblatt and S. 
Nellson, was able to demonstrate the first completely autonomous hand- 
eye system that could analyze a scene well enough to locate non- 
overlapping blocks and assemble them into a simple tower. In 1978 P. 
Uinston, S. Horn and E. Preudsr demonstrated their "copy" system, which 
looks at one scene of blocks and then assembles a copy of it from spare 
parte in another part of the table. 



THE ARTIFICIAL INTELLIGENCE LABORATORY PAGE 185 



Next, there was a period of activity, during uhich ue were publicly 
rather quiet, to rebuild the system. Our first system (and those of our 
colleagues at other centers) only "appeared" to see well; in fact they 
were Irapossibly sensitive to «isteke» because of occlusion of part of 
one object by another. Also, they Mere horribly sensitive to picture- 
processing errors due to not-qui te-perfect illumination, or flaws on the 
surfaces of the objects. It was only In the past year or so, with the 
completion of Uinston's heterarchical Vision System, and using 
Ingredients like the Guzman-Uinston-Wal tz object-finding theories and 
the Shirai heterarchical edge-ana tyser, that ue could get performance 
good enough to suggest noving on outside the closely controlled Blocks 
Uorid visual environment* The current systeu can cope uith very 
complicated occlusions of objects in three dimensions, using a single 
(non-stereoscopic) viewpoint. As noted in 15.11, ue plan to use range- 
finding and other more powerful "vision" methods in the more complicated 
real -wor Id appl i cat ions. 

F. LEARNING 

In the early years of our work, we tended to avoid attempting to moke 
machines that uere supposed to "learn to become smarter", following this 
principle: don't try to make a machine learn a class of behaviors until 
you are sure that the machine or procedural structures available are 
capable of supporting that behavior. This turned out to be a good idea, 
since little progress uas seen, in the first decade, by those attempting 
to get significant learned behavior by using Neural Network, Perceptron, 
Evolutionary, Adaptive-Adjustment, or Inductive-Inference theories. 

In fact, one of our most substantial theoretical accomplishments uas to 
develop the definitive theory presented in the book, PERCEPTRQNS, 
showing uhy certain lou-ievel linear optimization machine structures 
cannot learn to account for interactions of first level cues, features, 
or con text ^dependencies. 

By 1978 our general understanding of structural representations had 
progressed to the point of Winston's thesis, in uhich ue see a 
relatively high-level form of learning, in which uhat is learned depends 
on comparing descriptions of current events uith summary descriptions of 
uhat has com* before. This work has evoked widespread interest, and we 
see many projects, here and around the world, moving in that direction. 

Plany important things ue learn are not mere facts, but knouledge- 
handling capabi I i t ies» in particular, procedures that make for better 

learning strategies. The recent work of Goldstein and Sussman, uhich 
might appear to be concerned chiefly with Automatic Programming, is 
really concerned with modifying procedural representations of knowledge 

In accord with the extent to which the procedures In fact behave as they 
ar« "supposed" to. Ue regard this, then, as the nost proaising path 
presently available toward really "adaptive" machines. 
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G. COT1PUTER LANGUAGES 

The Al Laboratory has played an important roll in the evolution of the 
programming languages and computer systems that are used today by most 
successful workers in the AI field. The LISP system, originally 
developed by J. McCarthy, has become, in effect, the "standard" 
international language of AI. The Al Lab's LISP 1.6 system is probably 
the most powerful version of LISP, and is certainly in store extensive 
worldwide use than other variants- In its current form, one does not 
have to pay the "traditional** overhead of interpretative operation fo< 
procedures that do not require it; It is believed that its numerical 
efficiency is comparable to that of rao6t known FORTRAN or PL-1 
compilers! This LISP 1.6 has a powerful compiler, and an extremely 
versatile READ 'nput-output) system, both due largely to Jon Uhi te, 
exteneive interrupt system, and many other features. It meets practical 
compatibility requirements. The present version can run on any CEC 
18/58 monitor. Less sophisticated versions run on IBM systems* 

It wea discovered, however, that the data-structures originally provided 
in LISP were not really adequate for some of the "representations of 

knowledge" that AI was beginning to need. For example, Bobrow needed a 
form of "pattern-matching" for his linguistic system, dnd invented the 
language METEOR, embedded in LISP, so that he could have the features 

developed earlier by Yngve in his self-standing language, COfilT. At 

about the same time A. Guzman, working with H. Mcintosh developed a 
language CONVERT, also embedded in LISP, Initially for work In symbolic 
applied mathematics, a field that Mcintosh had conceived about the same 
time we did. Independently, the Newel l-Simon group at Carnegie also 

come to the conclusion that pattern-directed program control might make 
important advances in behavioral theories. 

None of those languages seemed to "stick". Perhaps they were 
inadequately engineered, perhaps they were before their time, certainly 
none of them had clearly formulated theoretical foundations of the 
quality that the original LISP of McCarthy had. But recently, the 
PLANNER proposal of Hewitt, as implemented in the MICRO-PLANNER language 
used by Uinograd land many others, now) seems to have met the real-world 
conditions for acceptance. Its descendant, CONNIVER. has recently had 
the most favorable acceptance in this family, but only the next few 
years will tell what is going to serve best. 

In other centers, new languages like QA-S, STRIPS, and SAIL Ar^ also 
moving in this direction ~ to make available pattern-directed control 
and to allow reference to multiple processes, multiple (bound) 
environments, "IF-NEEDEO" and "Antecedent-Theorem" demon-like processes 
and the like. These are all, we believe, reponses to the need for 
heterarchical control of knowledge-based problem-solving systems. 
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H, TIHE-SHAREO COMPUTER SYSTEM 

The HIT AI Laboratory developed the first tine-sharing system for* the 
PCP-1B (then POP-6) system. Thie system, called ITS, Has essentially 
completed about four years ago. It has recently become overloaded by 
the increasing complexity of the jobs it has to do. 

riany people agree that for AI purposes the ITS system is probably the 
most effective tine-sharing system availaole. Houever, because we do 
not want to be in the service business, ue are interested in accepting a 
larger system, even if not quite so effective, if the Maintenance can be 
provided elsewhere. However, this is not quite possible at present, ae 
will be noted in the section below about our computation requirements. 

I. CONTRIBUTIONS TO FUNDAMENTAL THEORIES 

The project hae made »any fundamental contributions both to Artificial 
Intelligence and to modern computer science in general. Ue will not 
review these in detail, but jutt Met their names. 

Mathenatical Theory of Cowputatloni J. McCarthy, et al. 

Theories of Schemata and Program Controls Heuitt, Patereon, et al. 

Abstract Complex) ty theory: Blum 

Perceptron Theory 

Concrete Complexity Theories: tlinsky, Papert 

Contributions to theory of parallel computers. 

Computational Geometry 

Contributions to theory of Productions, etc. 

J. COGNITIVE PSYCHOLOGY 

In the very last few years, it has become recognized that AI research ie 
much closer to psychological research than was at firet believed. The 
theories in the Vision System are widely considered ae candidates for 
theories of human vision. (See Sutherland's essay in the British SRC 
volume. ) 

The more general view, that what is learned is in large part determined 
by know* edge-based processes (as in Uinston's thesis) eee»s relatively 
new in psychology ™ although there is rarely anything really new in the 
sense that one can find such proposals in earlier Mterature — and is 
attracting a great deal of current attention. 

The Al-bdsed theories of education, as demonstrated in our LOGO project, 
have had a big Impact, recently, on the community of educational 
theorists. 
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K. AUTOMATIC PROGRAMMING 

This area is historically very close to AI. Indeed* the first really 
thoughtfully human-engineered debugging systems (like DDT) arose in the 
community loosely associated with AI work, J.C.R, Licklider was an 
early protnotor of such ideas. Ue believe that our recent work — 
notably of Sussoan and his colleagues -- has been the primary catalyst 
In the new wave of interest and optimism (e.g., as shown by the Balser 
report) . 

L. SPEECH RECOGNITION 

The revival of interest in this area for probably feasible applications 
stems perhaps as «uch fro* the advances in Al-types of heterarchica I 
(and semantic) programming as fro» any major advance in speech-science 
proper, 

M, OTHER SUBJECTS 

The complete summary of so large a Laboratory over so many years uould 
take too much space here- Us mention a few other topics of importance 
in their own right: 

Uork by Daniel Edwards on Cryptanalysis 

Uork by Bledsoe, Abrahams* Levin, Silver, Minsky, Norton, Slagie 

on theories of Mathematical Theorem-Proving 
Uork by Papert on theories of development of Intelligence 
Uork by A. L. Samuel of the well-known Checkers Program 
Uork by J, McCarthy on Chess 
Uork in many areas of applied aathematics %ry<i numerical 

analysis by R. Gosper, R. Schroeppel 

N. HARDWARE: OUR ttEMORY EMERGENCY 

The Laboratory needs an increase in computational power. Because of the 
knowledge-based character of the new generation of programs, their size 
is large compared to programs of five years ago. 

This is not a transient. All reasonably intelligent programs will be 
pretty large from now on. The practical cost of this largeness is not 
serious, except for the very immediate future, because all industry 
predictions agree that the cost of, say, a 589,889 word primary storage 
ui I i approach a few thousand dollars in the next decade. But right now 
we are in a crushing bind, because our time-sharing system can support 
only one such program at a time. 
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Higtoryi Ue contracted and supervised construction of the first fast 
mass core-store — through a developmsnt contract uith Fabri-tek. 
Although it was at first a powerful research tool, we have suffered 
since, by being "stuck" uith it — the reward of pioneering, ARPA has 
put aside our request for modernization from year to year. [t has 
become a critical bottleneck in achieving our goals. 

There are several problems uith it* 

(1) It is dangerously marginal. If there is a major failure, 
no one wit bs able to fix Itt It has been out of production 
for a long time and ue have maintained it ourselves. 

121 It is slow. Modern PDP-13's have 1 microsecond memories. 
The Fabri^tek memory is 2.9 nicroseconds. 

(3) It is too snail. Large programs saturate it and cause 
the system to go into "thrashing" mode of paging onto disc. 
MATHLAB. an offshoot of our Laboratory, has already found it 
necessary to add another 253X of memory. This has made it 
possible for them to run several knowledge-based programs at a 
time. Our system is swanped by one program of the size of 
Uinograd's Natural Language Program. Ue will find it very 
difficult to debug the application programs unless the memory 
is expanded. 

All of the system development has already been done, since the AI and 
I1ATHLAB machines use identical systems. The additional memory ul II also 

al lou Netuork users to try out our application programs. At present, 

this cannot be done. 

The system is extremely slow when any CONNIVER or similar program of the 
order of 10E3K is run, Ue use shared pure procedures as much as 
possible, and common compiled code, but the COWNIVER control stack must 
be represented as list-structure. 

SHROLU, P1c0ermott*s reasoning program. Levin's Vision program, and 
several other important research programs are too large to run on our 
current hardware when other users are competing for tine. To make 
reasonable progress, we would expect each of the project areas to have 1 
or 2 programs running at all times! 
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Doctoral Students fron the A, I , Laboratory 



The Laboratory has 
by this I isting of 



interesting academic 
of i ts doctoral dag 



record, as evidenced 
recipients. 



Slagle 


NIH. Hsad. A.I, Research group 


Jones 


NIH. 


Norton 


NIH, 


Hodes 


NIH. 


Blum 


Prof. Berkeley 


Raphael 


Head, robotics research, S.R. I. 


Bobrow 


Head, A.I. and language, Xerox (formerly BBfn 


" e ' t leman 


Xerox 


Abrahams 


Prof. N.Y.U. 


Winston 


Prof. MlT f Head of Robotics 


Ulnograd 


Prof. HIT (1973. Stanford) 


Hewitt 


Prof. HIT 


Horn 


Prof. HIT (1973) 


Krakauer 


Industry 


Gr i f t i th 


Industry (Information International ) 


Evans 


Industry (private software company) 


Guzman 


Prof, Poiytechnico, Hexico 


Uaitz 


Prof, Unlv of Illinois (1973) 


Charm ak 


Staff* Istituto Per Gli Stui Semantic! E 




Cogni tivi, Switzerland 


Hoses 


Prof. HIT Head of riATHLAB. MAC 


dart in 


Prof, MIT Head of Automatic Programming. MAC 


Luckham 


Prof, Stanford 


Smo 1 i ar 


Prof, Techn'ion (Israel), Univ. of Penn (1973) 


Sussnan 


Prof. HIT (1973) 


Goldstein 


Prof, tllT (1973) 


Aba 1 son 


Prof, HIT (1973) 


Perkins 


Lincoln Lab, 


Henneman 


Prof, 8*U, (Former ly.Univ of Texas) 


Beyer 


Prof, Unlv of Oregon 


Fal 1 


Prof, Northeastern Unlv. 



Other Ph,D. students who 
Baylor (from Carnegie) 
Bel ler (from Brand*!*) 
Roberts (from Carnegie) 



did their work at AI Lab, 



Ernst (IBH)i 
Waterloo); 



losely associated Ph.D. Theses; 
Roberts (AHPA); Sutherland (Utah)t Fischer 
KnoNlton (BTL) 
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Extended residencies* 

Bledsoe (Chairman, flathematics, Univ of Texas) 

Cocke (IBT1 research) 

Vogat (Geneva) Prof. CUNY Graduate Center 

ricConkie (Cornell) 

Marr (Cambridge) 

Paterson (Cambr i dge) 

Nevine (current) 

Rabin (Hebreu Univ*) 

Forte (Chairman, Yale Univ. Ilusic Oept*) 

Slawson (Chairman, llusic, Univ. Pit.) 

Binford (vision research, Stanford) 

Samuel (IBri Laboratories) 

Our Laboratory hat very close ties to the AI groups at 
Bolt, Beranek and Neusan 
Stanford 
Carneg i e-T1e I J on 
Stanford Research Institute 
Edinburgh 
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SECTION S 
SUTT1ARY PROGRESS REPORT 



HACHINE VISION 



Uork on machine vision has progressed rapidly in the last feu 
years. Many basic issues are now more sharply defined, 
permitting us to focus outside the restricted world of carefully 
prepared simple polyhedra. 

Ue here summarize some of the progress with emphasis of the last 
ytar. At the ■performance" level* we can take a collection of 
flat-sided objects of assorted shapes, pile them up, and ask the 
program to analyse, disassemble, and rearrange the objects into 
another structure* The latter can be specified by a symbolic 
description or by presenting a physical example to be analysed 
by the system. Many "low-level" vision problems had to be 
solved to reach this level of performance. Many of them are 
summarized in our January, 1972 Progress Report, and much more 
detail is available in technical notes and reports. Ue have 
made no compromises in our original long-term goal to set a firm 
foundation for Monocular Machine vision! This vision system 
works as yell on pictures of a scene as it does on the physical 
scene Itself, It ie not based on the use of physical range- 
finding methods, tactile-probe exploration, or other "active" 
sensors. 

The following particularly noteworthy results have appeared in 
the last year. 

1, David Ualtz has worked out a semantic theory of polyhedral line 
drawings that ie a major breakthrough in saveral respects. The 
theory gives deep Insights into the success of earlier uork and 
provides a powerful analysis capability for separating regions into 
bodies; in identifying edges as convex, concave, obscuring, shadow 
or crack; in using shadows to determine contact; and in reasoning 
out the orientation of object faces. 

2. Previous vision systems suffered froti an artificial division 
into I ine- finder/scene-ana lysis partnerships, communicating only by 
way of a handsd-over line drawing. The new systems of Jerry Lervan 
and Yoehiaki Shiral show how the barrier can be eliminated and how 
high level knowledge of physical constraints and partial analyst* 
can guide the filters and trackers that nost intimately deal with 
low- level in tens i ty information. 
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3. Tim Flnin has givtn the evolving vision system considerable 
deductive depth through several goal-oriented programs. One of 
these specializes in using a theory of "perceived groups". Often, 
some of an object'e Individual dimensions, position, or orientation 
parameters are indeterminate because of an obstruction in the line 
of sight. In these situations the vision system hypothesizes the 
missing information, using other objects considered similar by 
virtue of alignment in a stack, a common purpose, or simple 
proximi ty. 



4. Finin, Lerman, and Slesinger have completed a visual feedback 
module that checks the position of a block after positioning by the 
hand. Then it jiggles it into place if Its positional error 
exceeds a small threshold. This feedback link exploits the random* 
access capability of a programmable image acquisition system by 
looking only at points lying on a small circle around expected 



vertex locat Q -. ■ 



£. Bob Uoodham has done initial work on visual motion tracking. 
As a first step in effecting a coffee-pouring demonstration, he has 
worked out and compared sever: mechanisms for monitoring the 
rising lavs! of coffst in a stylised eup. 



5, Scott Fahlnan has devised a construction planning system which 
solves problems in two distinct directions. First, three 
dimensional modelling skill has been developed in the form of 
sophisticated touch and stability tests* Second, in cooperation 
with the specialists in COWNIVER language, he has demonstrated the 
need for and use of advanced control and data base mechanisms* The 
eystem can plan fairly complicated constructions requiring 
temporary scaffolding supports or counterweights. 



7. Rich Boberg has explored the problem of reversing the analyeis 
process, that is, reconstructing a scene from an abstract 
description. Lie believe this is the first step toward an automatic 
design system where the machine contains and uses considerable 
common sense knowledge about the constraints inherent in a physical 
wor d< 



8. John Holierbach has probed the problem of describing complex 
shapes through work on complicated, higher order polyhedra. His 
heuristic theory of projection shows how many objects can be 
sensibly decomposed into basic shapes, modified by protrusions and 
indentations. 
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9. In another domain, "ark Adler has shown how to make progress 
toward solving the problem of lint drawing* with curves- In a 
style reminiscent of initial work on polyhedra, he has outlined an 
approach to the analysis of some highly constrained kinds of 
drawings. This should contribute conceptually to work on more 
general real vision, to diagram reading and Manipulating services* 
and eventually to personal assistant systems In which sketches must 
supplement commands in natural language that do not lend themselves 
to verbal explanations. 

PROGRESS IN MACHINE LEARNING 

The long sought goal of machine learning has seen a major conceptual 
breakthrough. The concept Is simple! we say that someone has learned 
eomething if he is able to perform appropriate processes- One rarely 
builds a new process from nothingi presumably one extends and adapts 
processes developed for other goals. Thus, learning is like debugging a 
computer program, and a smart person is one who knows good ways to 
characterize defects and requirements, and has good methods for making 

the appropriate procedure changes. This idea, in different ways, has 

led to the following large steps. 

1* G. Sussaan has completed a computer program that contains some 
sophisticated knowledge about diagnosis and repair of computer 
programs. Given a sequence of blocks-building tasks, similar to 
those performed by Uinograd's natural language program, Sussman's 
program performs a sequence of modifications on an Initially 
trivial building program. Eventually, the "learned" procedure is 
able to perform all the operations that were Initially built-into 
Uinograd's system. Sussman's methods have attracted wide 
attention, even before publication of his thesis, and are the basis 
of a number of other automatic programming proposals. 

2* I* Goldstein has developed a theory of automatic analysis and 
systemat izat ion of programs which propose to achieve certain kinde 
of goals but do not actually work. By setting up an "annotation" 
structure based on matching parts of the defective procedure to 
parts of the goal description, Goldstein's procedure can propose 
and make corrections, using methods similar to those of Sussman. 
Again, part of the "secret" of learning lies in having the 
knowledge and ability to focus clearly on what parts of procedures 
are not working properly, and Goldstein's thesis is, we believe, a 
major move in this direction. 

3. 11. flinsky has formulateo a new theory, called Frame-Systems, 
which, it is hoped, will show quickly how to do complicated common- 
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sense reasoning In systems that are not confused by containing 
large amounts of non-relevant information. This theory also 
suggest* a number of ways in uhlch new knowledge can be added in an 
orderly way. again without leading to information overloads* It is 
hoped that this theory will combine with Winston's earlier learning 
program ideas to allow operation over a wider range of tasks* 



IHCRO-AUTONATION 

Ue have now selected and acquired a computer conf igurat ion, a vidicon 
system* and a digitally driven x-y table. A new arm designed for us is 
under construction for November 1973. A new computer sye design is 
being completed. It is expected to have much better optical properties 
than previ ous computer 



AUTOMATIC PROGRAMING 

Ue mentioned above the projects of Goldstein and Sussmani Program 
Analysis* and Automatic Debugging. Another project which is well under 
uay» concerned with developing a programming formalism and technology. 
Is the ACTOR system of Hewitt and his associates* in which implicit and 
undesirable interactions are unlikely to arise accidentally. 

NATURAL LANGUAGE RESEARCH 

Uinograd's BLOCKS program demonstrated nsw dinensions to the connection* 
between details of the structure of natural English and the meanings of 
words, clauses, and whole discourses. Our experience with SHRDLU hae 
had two important consequsncssi it has shown us that quite non-trivial 
natural language programs can be written with today's hardware and 
software, and this In turn hae encouraged a fresh burst of natural 
language research, not only hmrm but in other laboratories. 

The current proposal summarizes recent work on consolidating what was 
learned fro* this work, and what has bssn done with it recently. The 
eystem has been revised and documented so that it can be used by others 
who want to go further in this area. 



